tribuchet: building on jamie Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.01 seconds) Test will time out and terminate in 3600 seconds run the VM test script start all VMs gitlab: starting vm gitlab # Disk image does not exist, creating the virtualisation disk image... gitlab: QEMU running (pid 46) gitlab # Formatting '/build/vm-state-gitlab/tmp.tF4AS0yiha', fmt=raw size=1073741824 gitlab # mke2fs 1.47.4 (6-Mar-2025) gitlab # Discarding device blocks: 0/262144 done gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes gitlab # Filesystem UUID: 298345a3-4800-42c2-91e8-b535d43c2161 gitlab # Superblock backups stored on blocks: (finished: start all VMs, in 49.48 seconds) gitlab: waiting for unit setup-gitlab.service gitlab # 32768, 98304, 163840, 229376 gitlab: waiting for the VM to finish booting gitlab # gitlab # Allocating group tables: 0/8 done gitlab # Writing inode tables: 0/8 done gitlab # Creating journal (8192 blocks): done gitlab # Writing superblocks and filesystem accounting information: 0/8 done gitlab # gitlab # Virtualisation disk image created. gitlab # Creating Nix store image... gitlab # Created Nix store image. gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)... gitlab # gitlab # gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)... gitlab # gitlab # gitlab # Booting from ROM... gitlab # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 6 17:27:09 UTC 2026 gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/fa7yp7hwl9zp8x7hiqk233hjsnpm2rr4-nixos-system-gitlab-test/init regInfo=/nix/store/0g58cgipik8h01w6dbqcc3xv39xxdyjh-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.000000] BIOS-provided physical RAM map: gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved gitlab # [ 0.000000] NX (Execute Disable) protection: active gitlab # [ 0.000000] APIC: Static calls initialized gitlab # [ 0.000000] SMBIOS 2.8 present. gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 gitlab # [ 0.000000] DMI: Memory slots populated: 1/1 gitlab # [ 0.000000] Hypervisor detected: KVM gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 gitlab # [ 0.000000] kvm-clock: using sched offset of 474553289 cycles gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns gitlab # [ 0.000004] tsc: Detected 2400.012 MHz processor gitlab # [ 0.000803] last_pfn = 0x1c0000 max_arch_pfn = 0x10000000000 gitlab # [ 0.000830] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs gitlab # [ 0.000832] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT gitlab # [ 0.000870] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.002722] found SMP MP-table at [mem 0x000f5470-0x000f547f] gitlab # [ 0.002733] Using GB pages for direct mapping gitlab # [ 0.002801] RAMDISK: [mem 0xbe31c000-0xbffcffff] gitlab # [ 0.002808] ACPI: Early table checksum verification disabled gitlab # [ 0.002811] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) gitlab # [ 0.002815] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002818] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002826] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002828] ACPI: FACS 0x00000000BFFE0000 000040 gitlab # [ 0.002830] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002831] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002833] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002834] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2] gitlab # [ 0.002835] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e] gitlab # [ 0.002836] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] gitlab # [ 0.002836] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482] gitlab # [ 0.002837] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba] gitlab # [ 0.002838] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2] gitlab # [ 0.003079] No NUMA configuration found gitlab # [ 0.003080] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff] gitlab # [ 0.003083] NODE_DATA(0) allocated [mem 0x1bfffa780-0x1bffffcff] gitlab # [ 0.005470] Zone ranges: gitlab # [ 0.005471] DMA [mem 0x0000000000001000-0x0000000000ffffff] gitlab # [ 0.005473] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] gitlab # [ 0.005474] Normal [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.005475] Device empty gitlab # [ 0.005475] Movable zone start for each node gitlab # [ 0.005476] Early memory node ranges gitlab # [ 0.005476] node 0: [mem 0x0000000000001000-0x000000000009efff] gitlab # [ 0.005477] node 0: [mem 0x0000000000100000-0x00000000bffd7fff] gitlab # [ 0.005478] node 0: [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.005479] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff] gitlab # [ 0.005499] On node 0, zone DMA: 1 pages in unavailable ranges gitlab # [ 0.005775] On node 0, zone DMA: 97 pages in unavailable ranges gitlab # [ 0.120102] On node 0, zone Normal: 40 pages in unavailable ranges gitlab # [ 0.120536] ACPI: PM-Timer IO Port: 0x608 gitlab # [ 0.120550] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) gitlab # [ 0.120581] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 gitlab # [ 0.120584] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) gitlab # [ 0.120586] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) gitlab # [ 0.120587] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) gitlab # [ 0.120588] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) gitlab # [ 0.120589] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) gitlab # [ 0.120592] ACPI: Using ACPI (MADT) for SMP configuration information gitlab # [ 0.120593] ACPI: HPET id: 0x8086a201 base: 0xfed00000 gitlab # [ 0.120597] TSC deadline timer available gitlab # [ 0.120602] CPU topo: Max. logical packages: 1 gitlab # [ 0.120603] CPU topo: Max. logical dies: 1 gitlab # [ 0.120603] CPU topo: Max. dies per package: 1 gitlab # [ 0.120607] CPU topo: Max. threads per core: 1 gitlab # [ 0.120608] CPU topo: Num. cores per package: 4 gitlab # [ 0.120608] CPU topo: Num. threads per package: 4 gitlab # [ 0.120608] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs gitlab # [ 0.120632] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() gitlab # [ 0.120648] kvm-guest: KVM setup pv remote TLB flush gitlab # [ 0.120651] kvm-guest: setup PV sched yield gitlab # [ 0.120668] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] gitlab # [ 0.120669] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] gitlab # [ 0.120670] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff] gitlab # [ 0.120672] [mem 0xc0000000-0xfeffbfff] available for PCI devices gitlab # [ 0.120673] Booting paravirtualized kernel on KVM gitlab # [ 0.120677] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns gitlab # [ 0.125145] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 gitlab # [ 0.127391] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288 gitlab # [ 0.127438] kvm-guest: PV spinlocks enabled gitlab # [ 0.127440] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) gitlab # [ 0.127443] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/fa7yp7hwl9zp8x7hiqk233hjsnpm2rr4-nixos-system-gitlab-test/init regInfo=/nix/store/0g58cgipik8h01w6dbqcc3xv39xxdyjh-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.127537] Unknown kernel command line parameters "regInfo=/nix/store/0g58cgipik8h01w6dbqcc3xv39xxdyjh-closure-info/registration", will be passed to user space. gitlab # [ 0.127550] random: crng init done gitlab # [ 0.127551] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes gitlab # [ 0.136647] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) gitlab # [ 0.141352] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) gitlab # [ 0.141431] software IO TLB: area num 4. gitlab # [ 0.216332] Fallback order for Node 0: 0 gitlab # [ 0.216339] Built 1 zonelists, mobility grouping on. Total pages: 1572726 gitlab # [ 0.216341] Policy zone: Normal gitlab # [ 0.218863] mem auto-init: stack:all(zero), heap alloc:on, heap free:off gitlab # [ 0.228461] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 gitlab # [ 0.242387] allocated 12582912 bytes of page_ext gitlab # [ 0.252424] ftrace: allocating 48717 entries in 192 pages gitlab # [ 0.252426] ftrace: allocated 192 pages with 2 groups gitlab # [ 0.253288] Dynamic Preempt: lazy gitlab # [ 0.253471] rcu: Preemptible hierarchical RCU implementation. gitlab # [ 0.253471] rcu: RCU event tracing is enabled. gitlab # [ 0.253472] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. gitlab # [ 0.253474] Trampoline variant of Tasks RCU enabled. gitlab # [ 0.253474] Rude variant of Tasks RCU enabled. gitlab # [ 0.253474] Tracing variant of Tasks RCU enabled. gitlab # [ 0.253475] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. gitlab # [ 0.253476] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 gitlab # [ 0.253501] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.253503] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.253504] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.257865] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16 gitlab # [ 0.258185] rcu: srcu_init: Setting srcu_struct sizes based on contention. gitlab # [ 0.258193] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns gitlab # [ 0.258306] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) gitlab # [ 0.261903] Console: colour VGA+ 80x25 gitlab # [ 0.261906] printk: legacy console [tty0] enabled gitlab # [ 0.292689] printk: legacy console [ttyS0] enabled gitlab # [ 0.400107] ACPI: Core revision 20250807 gitlab # [ 0.401079] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns gitlab # [ 0.402727] APIC: Switch to symmetric I/O mode setup gitlab # [ 0.403784] x2apic enabled gitlab # [ 0.404538] APIC: Switched APIC routing to: physical x2apic gitlab # [ 0.405513] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask() gitlab # [ 0.406791] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself() gitlab # [ 0.408304] kvm-guest: setup PV IPIs gitlab # [ 0.409910] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 gitlab # [ 0.410991] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns gitlab # [ 0.412815] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012) gitlab # [ 0.413901] x86/cpu: User Mode Instruction Prevention (UMIP) activated gitlab # [ 0.414937] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 gitlab # [ 0.416813] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 gitlab # [ 0.417817] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto gitlab # [ 0.418813] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl gitlab # [ 0.419813] Transient Scheduler Attacks: Vulnerable: No microcode gitlab # [ 0.421607] Spectre V2 : Mitigation: Enhanced / Automatic IBRS gitlab # [ 0.422569] Speculative Return Stack Overflow: Mitigation: Safe RET gitlab # [ 0.423624] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization gitlab # [ 0.424820] Spectre V2 : Enabling IBPB for BPF gitlab # [ 0.425814] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier gitlab # [ 0.426814] active return thunk: srso_alias_return_thunk gitlab # [ 0.427835] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' gitlab # [ 0.428813] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' gitlab # [ 0.430813] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' gitlab # [ 0.431813] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' gitlab # [ 0.432813] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' gitlab # [ 0.433813] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' gitlab # [ 0.434813] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' gitlab # [ 0.435813] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 gitlab # [ 0.437618] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 gitlab # [ 0.438614] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 gitlab # [ 0.439607] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 gitlab # [ 0.440616] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 gitlab # [ 0.441617] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format. gitlab # [ 0.471675] Freeing SMP alternatives memory: 44K gitlab # [ 0.472432] pid_max: default: 32768 minimum: 301 gitlab # [ 0.472913] LSM: initializing lsm=capability,landlock,yama,bpf,ima gitlab # [ 0.473925] landlock: Up and running. gitlab # [ 0.475292] Yama: becoming mindful. gitlab # [ 0.476033] LSM support for eBPF active gitlab # [ 0.477046] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.477964] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.481199] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) gitlab # [ 0.482406] Performance Events: Fam17h+ core perfctr, AMD PMU driver. gitlab # [ 0.482822] ... version: 2 gitlab # [ 0.483614] ... bit width: 48 gitlab # [ 0.483815] ... generic counters: 6 gitlab # [ 0.484587] ... generic bitmap: 000000000000003f gitlab # [ 0.484816] ... fixed-purpose counters: 0 gitlab # [ 0.485560] ... fixed-purpose bitmap: 0000000000000000 gitlab # [ 0.485815] ... value mask: 0000ffffffffffff gitlab # [ 0.486775] ... max period: 00007fffffffffff gitlab # [ 0.487573] ... global_ctrl mask: 000000000000003f gitlab # [ 0.487917] signal: max sigframe size: 3376 gitlab # [ 0.488762] rcu: Hierarchical SRCU implementation. gitlab # [ 0.489473] rcu: Max phase no-delay instances is 400. gitlab # [ 0.490040] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level gitlab # [ 0.495547] smp: Bringing up secondary CPUs ... gitlab # [ 0.496235] smpboot: x86: Booting SMP configuration: gitlab # [ 0.496824] .... node #0, CPUs: #1 #2 #3 gitlab # [ 0.499984] smp: Brought up 1 node, 4 CPUs gitlab # [ 0.501569] smpboot: Total of 4 processors activated (19200.09 BogoMIPS) gitlab # [ 0.502324] Memory: 6011568K/6290904K available (17210K kernel code, 2726K rwdata, 13576K rodata, 3640K init, 2992K bss, 260100K reserved, 0K cma-reserved) gitlab # [ 0.503816] devtmpfs: initialized gitlab # [ 0.504895] x86/mm: Memory block size: 128MB gitlab # [ 0.507993] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) gitlab # [ 0.509893] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). gitlab # [ 0.510912] pinctrl core: initialized pinctrl subsystem gitlab # [ 0.512100] PM: RTC time: 18:51:17, date: 2026-10-04 gitlab # [ 0.515750] NET: Registered PF_NETLINK/PF_ROUTE protocol family gitlab # [ 0.518075] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations gitlab # [ 0.518871] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations gitlab # [ 0.520922] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations gitlab # [ 0.521827] audit: initializing netlink subsys (disabled) gitlab # [ 0.522919] audit: type=2000 audit(1791139877.009:1): state=initialized audit_enabled=0 res=1 gitlab # [ 0.523075] thermal_sys: Registered thermal governor 'fair_share' gitlab # [ 0.524817] thermal_sys: Registered thermal governor 'bang_bang' gitlab # [ 0.525816] thermal_sys: Registered thermal governor 'step_wise' gitlab # [ 0.526816] thermal_sys: Registered thermal governor 'user_space' gitlab # [ 0.527815] thermal_sys: Registered thermal governor 'power_allocator' gitlab # [ 0.528860] cpuidle: using governor menu gitlab # [ 0.531047] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 gitlab # [ 0.532061] PCI: Using configuration type 1 for base access gitlab # [ 0.532816] PCI: Using configuration type 1 for extended access gitlab # [ 0.534012] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. gitlab # [ 0.536056] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages gitlab # [ 0.537740] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page gitlab # [ 0.538687] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages gitlab # [ 0.539692] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page gitlab # [ 0.545440] ACPI: Added _OSI(Module Device) gitlab # [ 0.546826] ACPI: Added _OSI(Processor Device) gitlab # [ 0.548823] ACPI: Added _OSI(Processor Aggregator Device) gitlab # [ 0.556863] ACPI: 1 ACPI AML tables successfully acquired and loaded gitlab # [ 0.563095] ACPI: Interpreter enabled gitlab # [ 0.563837] ACPI: PM: (supports S0 S3 S4 S5) gitlab # [ 0.564640] ACPI: Using IOAPIC for interrupt routing gitlab # [ 0.565510] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug gitlab # [ 0.568824] PCI: Using E820 reservations for host bridge windows gitlab # [ 0.571273] ACPI: Enabled 2 GPEs in block 00 to 0F gitlab # [ 0.585240] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) gitlab # [ 0.587834] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] gitlab # [ 0.592923] acpiphp: Slot [3] registered gitlab # [ 0.593875] acpiphp: Slot [4] registered gitlab # [ 0.595873] acpiphp: Slot [5] registered gitlab # [ 0.597874] acpiphp: Slot [6] registered gitlab # [ 0.598900] acpiphp: Slot [7] registered gitlab # [ 0.600863] acpiphp: Slot [8] registered gitlab # [ 0.601861] acpiphp: Slot [9] registered gitlab # [ 0.603870] acpiphp: Slot [10] registered gitlab # [ 0.604882] acpiphp: Slot [11] registered gitlab # [ 0.606863] acpiphp: Slot [12] registered gitlab # [ 0.607861] acpiphp: Slot [13] registered gitlab # [ 0.608872] acpiphp: Slot [14] registered gitlab # [ 0.610878] acpiphp: Slot [15] registered gitlab # [ 0.611860] acpiphp: Slot [16] registered gitlab # [ 0.613860] acpiphp: Slot [17] registered gitlab # [ 0.614877] acpiphp: Slot [18] registered gitlab # [ 0.616862] acpiphp: Slot [19] registered gitlab # [ 0.617837] acpiphp: Slot [20] registered gitlab # [ 0.618576] acpiphp: Slot [21] registered gitlab # [ 0.618846] acpiphp: Slot [22] registered gitlab # [ 0.619549] acpiphp: Slot [23] registered gitlab # [ 0.620836] acpiphp: Slot [24] registered gitlab # [ 0.621553] acpiphp: Slot [25] registered gitlab # [ 0.621849] acpiphp: Slot [26] registered gitlab # [ 0.622578] acpiphp: Slot [27] registered gitlab # [ 0.623837] acpiphp: Slot [28] registered gitlab # [ 0.624525] acpiphp: Slot [29] registered gitlab # [ 0.624851] acpiphp: Slot [30] registered gitlab # [ 0.625603] acpiphp: Slot [31] registered gitlab # [ 0.625829] PCI host bridge to bus 0000:00 gitlab # [ 0.626534] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] gitlab # [ 0.627828] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] gitlab # [ 0.628816] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] gitlab # [ 0.630817] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] gitlab # [ 0.631816] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 0.632985] pci_bus 0000:00: root bus resource [bus 00-ff] gitlab # [ 0.637094] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint gitlab # [ 0.642461] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint gitlab # [ 0.646920] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint gitlab # [ 0.653910] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f] gitlab # [ 0.655890] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk gitlab # [ 0.658822] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk gitlab # [ 0.660822] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk gitlab # [ 0.663822] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk gitlab # [ 0.666323] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint gitlab # [ 0.672919] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f] gitlab # [ 0.677365] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint gitlab # [ 0.681526] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI gitlab # [ 0.684834] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB gitlab # [ 0.687279] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint gitlab # [ 0.713962] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] gitlab # [ 0.714834] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] gitlab # [ 0.715839] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref] gitlab # [ 0.717419] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] gitlab # [ 0.719965] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.725859] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f] gitlab # [ 0.726783] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] gitlab # [ 0.727632] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref] gitlab # [ 0.740824] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] gitlab # [ 0.743777] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.748856] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f] gitlab # [ 0.749763] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] gitlab # [ 0.750667] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref] gitlab # [ 0.753062] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.757854] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf] gitlab # [ 0.758781] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] gitlab # [ 0.759693] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref] gitlab # [ 0.763101] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.766856] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df] gitlab # [ 0.768850] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] gitlab # [ 0.769868] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref] gitlab # [ 0.772044] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.775854] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f] gitlab # [ 0.776816] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] gitlab # [ 0.777838] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref] gitlab # [ 0.781378] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.787855] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff] gitlab # [ 0.788807] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] gitlab # [ 0.790626] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref] gitlab # [ 0.793121] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.797849] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff] gitlab # [ 0.798778] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] gitlab # [ 0.799624] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref] gitlab # [ 0.801824] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref] gitlab # [ 0.804130] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint gitlab # [ 0.806855] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] gitlab # [ 0.807839] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref] gitlab # [ 0.810119] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint gitlab # [ 0.813860] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f] gitlab # [ 0.814823] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] gitlab # [ 0.815816] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref] gitlab # [ 0.819082] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.823850] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f] gitlab # [ 0.824805] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff] gitlab # [ 0.825630] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref] gitlab # [ 0.830629] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 gitlab # [ 0.831788] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 gitlab # [ 0.833969] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 gitlab # [ 0.834953] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 gitlab # [ 0.835896] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 gitlab # [ 0.838938] iommu: Default domain type: Translated gitlab # [ 0.839819] iommu: DMA domain TLB invalidation policy: lazy mode gitlab # [ 0.841076] ACPI: bus type USB registered gitlab # [ 0.841894] usbcore: registered new interface driver usbfs gitlab # [ 0.842868] usbcore: registered new interface driver hub gitlab # [ 0.843851] usbcore: registered new device driver usb gitlab # [ 0.846527] NetLabel: Initializing gitlab # [ 0.847817] NetLabel: domain hash size = 128 gitlab # [ 0.848629] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO gitlab # [ 0.849697] NetLabel: unlabeled traffic allowed by default gitlab # [ 0.850594] PCI: Using ACPI for IRQ routing gitlab # [ 0.852028] pci 0000:00:02.0: vgaarb: setting as boot VGA device gitlab # [ 0.852812] pci 0000:00:02.0: vgaarb: bridge control possible gitlab # [ 0.852812] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none gitlab # [ 0.854821] vgaarb: loaded gitlab # [ 0.855614] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 gitlab # [ 0.856472] hpet0: 3 comparators, 64-bit 100.000000 MHz counter gitlab # [ 0.861956] clocksource: Switched to clocksource kvm-clock gitlab # [ 0.865976] VFS: Disk quotas dquot_6.6.0 gitlab # [ 0.866756] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) gitlab # [ 0.868530] pnp: PnP ACPI init gitlab # [ 0.869866] pnp: PnP ACPI: found 6 devices gitlab # [ 0.882980] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns gitlab # [ 0.886863] clocksource: Switched to clocksource acpi_pm gitlab # [ 0.889701] NET: Registered PF_INET protocol family gitlab # [ 0.894940] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) gitlab # [ 0.930010] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) gitlab # [ 0.931726] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) gitlab # [ 0.933204] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) gitlab # [ 0.937534] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) gitlab # [ 0.939042] TCP: Hash tables configured (established 65536 bind 65536) gitlab # [ 0.940606] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) gitlab # [ 0.942325] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.943888] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.945400] NET: Registered PF_UNIX/PF_LOCAL protocol family gitlab # [ 0.946547] NET: Registered PF_XDP protocol family gitlab # [ 0.947518] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] gitlab # [ 0.948591] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] gitlab # [ 0.949794] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] gitlab # [ 0.951058] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] gitlab # [ 0.952234] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 0.953715] pci 0000:00:01.0: PIIX3: Enabling Passive Release gitlab # [ 0.954754] pci 0000:00:00.0: Limiting direct PCI/PCI transfers gitlab # [ 0.957370] ACPI: \_SB_.LNKD: Enabled at IRQ 11 gitlab # [ 0.960301] PCI: CLS 0 bytes, default 64 gitlab # [ 0.961143] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) gitlab # [ 0.961605] Trying to unpack rootfs image as initramfs... gitlab # [ 0.962384] software IO TLB: mapped [mem 0x00000000ba31c000-0x00000000be31c000] (64MB) gitlab # [ 0.965353] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns gitlab # [ 0.986859] Initialise system trusted keyrings gitlab # [ 0.988158] workingset: timestamp_bits=40 max_order=21 bucket_order=0 gitlab # [ 0.999898] Key type asymmetric registered gitlab # [ 1.000671] Asymmetric key parser 'x509' registered gitlab # [ 1.001553] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) gitlab # [ 1.003534] io scheduler mq-deadline registered gitlab # [ 1.004356] io scheduler kyber registered gitlab # [ 1.010457] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled gitlab # [ 1.011949] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A gitlab # [ 1.014492] Linux agpgart interface v0.103 gitlab # [ 1.015379] ACPI: bus type drm_connector registered gitlab # [ 1.018602] usbcore: registered new interface driver usbserial_generic gitlab # [ 1.019779] usbserial: USB Serial support registered for generic gitlab # [ 1.020890] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. gitlab # [ 1.020890] Please enable it if your BIOS has the CPPC option. gitlab # [ 1.023272] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled gitlab # [ 1.024811] drop_monitor: Initializing network drop monitor service gitlab # [ 1.026051] NET: Registered PF_INET6 protocol family gitlab # [ 1.027952] Segment Routing with IPv6 gitlab # [ 1.028711] In-situ OAM (IOAM) with IPv6 gitlab # [ 1.031371] IPI shorthand broadcast: enabled gitlab # [ 1.036000] sched_clock: Marking stable (889017650, 146619167)->(1058914516, -23277699) gitlab # [ 1.037932] registered taskstats version 1 gitlab # [ 1.039062] Loading compiled-in X.509 certificates gitlab # [ 1.049399] Demotion targets for Node 0: null gitlab # [ 1.050773] Key type .fscrypt registered gitlab # [ 1.051506] Key type fscrypt-provisioning registered gitlab # [ 1.052542] ima: No TPM chip found, activating TPM-bypass! gitlab # [ 1.053553] ima: Allocated hash algorithm: sha1 gitlab # [ 1.054479] ima: No architecture policies found gitlab # [ 1.055575] PM: Magic number: 14:573:897 gitlab # [ 1.057433] RAS: Correctable Errors collector initialized. gitlab # [ 1.062163] clk: Disabling unused clocks gitlab # [ 1.062948] PM: genpd: Disabling unused power domains gitlab # [ 1.267723] Freeing initrd memory: 29392K gitlab # [ 1.276445] Freeing unused decrypted memory: 2028K gitlab # [ 1.283357] Freeing unused kernel image (initmem) memory: 3640K gitlab # [ 1.286160] Write protecting the kernel read-only data: 32768k gitlab # [ 1.291325] Freeing unused kernel image (text/rodata gap) memory: 1220K gitlab # [ 1.295353] Freeing unused kernel image (rodata/data gap) memory: 760K gitlab # [ 1.363419] x86/mm: Checked W+X mappings: passed, no W+X pages found. gitlab # [ 1.364729] Run /init as init process gitlab # [ 1.378538] systemd[1]: Inserted module 'autofs4' gitlab # [ 1.401785] fuse: init (API version 7.45) gitlab # [ 1.410393] ACPI: \_SB_.LNKC: Enabled at IRQ 10 gitlab # [ 1.417552] ACPI: \_SB_.LNKA: Enabled at IRQ 10 gitlab # [ 1.420941] ACPI: \_SB_.LNKB: Enabled at IRQ 11 gitlab # [ 1.460810] systemd[1]: Successfully made /usr/ read-only. gitlab # [ 1.797595] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 1.838267] systemd[1]: Detected virtualization kvm. gitlab # [ 1.840535] systemd[1]: Detected architecture x86-64. gitlab # [ 1.842818] systemd[1]: Running in initrd. gitlab # [ 1.845519] systemd[1]: Initializing machine ID from random generator. gitlab # [ 1.848536] systemd[1]: Hostname set to . gitlab # [ 1.962268] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 2.006872] systemd[1]: Queued start job for default target Initrd Default Target. gitlab # [ 2.022550] systemd[1]: Created slice Slice /system/modprobe. gitlab # [ 2.023768] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 2.025220] systemd[1]: Expecting device /dev/disk/by-label/nix-store... gitlab # [ 2.026411] systemd[1]: Expecting device /dev/disk/by-label/nixos... gitlab # [ 2.027579] systemd[1]: Reached target Path Units. gitlab # [ 2.028531] systemd[1]: Reached target Slice Units. gitlab # [ 2.029446] systemd[1]: Reached target Swaps. gitlab # [ 2.030277] systemd[1]: Reached target Timer Units. gitlab # [ 2.031301] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 2.032549] systemd[1]: Listening on Journal Socket (/dev/log). gitlab # [ 2.033783] systemd[1]: Listening on Journal Sockets. gitlab # [ 2.034835] systemd[1]: Listening on udev Control Socket. gitlab # [ 2.035935] systemd[1]: Listening on udev Kernel Socket. gitlab # [ 2.036940] systemd[1]: Reached target Socket Units. gitlab # [ 2.039118] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 2.041294] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 2.043466] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 2.053936] systemd[1]: Starting Journal Service... gitlab # [ 2.059837] systemd[1]: Starting Load Kernel Modules... gitlab # [ 2.062305] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 2.068281] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 2.078147] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 2.087762] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. gitlab # [ 2.088282] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 2.092702] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev gitlab # [ 2.099428] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 2.099851] systemd-journald[93]: Collecting audit messages is disabled. gitlab # [ 2.100761] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config gitlab # [ 2.116754] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 2.122713] netfs: FS-Cache loaded gitlab # [ 2.123372] systemd[1]: Finished Load Kernel Modules. gitlab # [ 2.129929] 9pnet: Installing 9P2000 support gitlab # [ 2.134394] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 2.136604] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 2.138499] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 2.142002] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 2.145719] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 2.147541] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 2.007371] systemd-modules-load[94]: Using 4 probe threads gitlab # [ 2.154760] systemd[1]: Started Journal Service. gitlab # [ 2.009777] systemd-modules-load[94]: Inserted module 'virtio_balloon' gitlab # [ 2.010951] systemd-modules-load[94]: Inserted module 'dm_mod' gitlab # [ 2.012527] systemd-modules-load[94]: Inserted module 'virtio_gpu' gitlab # [ 2.015411] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 2.017554] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 2.018692] systemd[1]: Reached target Local File Systems. gitlab # [ 2.019573] systemd[1]: Starting Create System Files and Directories... gitlab # [ 2.020571] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 2.030502] systemd[1]: Finished Create System Files and Directories. gitlab # [ 2.042633] systemd-udevd[115]: Using default interface naming scheme 'v261'. gitlab # [ 2.053901] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 2.080749] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 2.082161] systemd[1]: Reached target System Initialization. gitlab # [ 2.083244] systemd[1]: Reached target Basic System. gitlab # [ 2.309726] uhci_hcd 0000:00:01.2: UHCI Host Controller gitlab # [ 2.310610] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1 gitlab # [ 2.311760] uhci_hcd 0000:00:01.2: detected 2 ports gitlab # [ 2.312555] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140 gitlab # [ 2.313716] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 gitlab # [ 2.314963] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 gitlab # [ 2.316044] usb usb1: Product: UHCI Host Controller gitlab # [ 2.317311] usb usb1: Manufacturer: Linux 6.18.43 uhci_hcd gitlab # [ 2.318130] usb usb1: SerialNumber: 0000:00:01.2 gitlab # [ 2.328493] SCSI subsystem initialized gitlab # [ 2.347151] virtio_blk virtio4: 4/0/0 default/read/poll queues gitlab # [ 2.353820] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 gitlab # [ 2.354990] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) gitlab # [ 2.359882] hub 1-0:1.0: USB hub found gitlab # [ 2.360654] hub 1-0:1.0: 2 ports detected gitlab # [ 2.361412] virtio_blk virtio5: 4/0/0 default/read/poll queues gitlab # [ 2.365375] serio: i8042 KBD port at 0x60,0x64 irq 1 gitlab # [ 2.366279] serio: i8042 AUX port at 0x60,0x64 irq 12 gitlab # [ 2.367287] virtio_blk virtio5: [vdb] 11952888 512-byte logical blocks (6.12 GB/5.70 GiB) gitlab # [ 2.261520] (udev-worker)[134]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 2.263743] (udev-worker)[134]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.266755] systemd[1]: Starting Virtual Console Setup... gitlab # [ 2.415425] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 gitlab # [ 2.417837] scsi host0: ata_piix gitlab # [ 2.278595] (udev-worker)[119]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.284728] systemd[1]: Found device /dev/disk/by-label/nix-store. gitlab # [ 2.433219] scsi host1: ata_piix gitlab # [ 2.433973] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0 gitlab # [ 2.435296] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0 gitlab # [ 2.291153] systemd-vconsole-setup[150]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 2.295296] systemd[1]: Found device /dev/disk/by-label/nixos. gitlab # [ 2.296278] systemd[1]: Finished Virtual Console Setup. gitlab # [ 2.297377] systemd[1]: Reached target Initrd Root Device. gitlab # [ 2.298255] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... gitlab # [ 2.322264] systemd-fsck[161]: nixos: clean, 12/65536 files, 13019/262144 blocks gitlab # [ 2.325819] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. gitlab # [ 2.581970] usb 1-1: new full-speed USB device number 2 using uhci_hcd gitlab # [ 2.597262] ata2: found unknown device (class 0) gitlab # [ 2.600037] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 gitlab # [ 2.604542] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 gitlab # [ 2.666872] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray gitlab # [ 2.694288] cdrom: Uniform CD-ROM driver Revision: 3.20 gitlab # [ 2.751550] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 gitlab # [ 2.754699] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 gitlab # [ 2.757439] usb 1-1: Product: QEMU USB Tablet gitlab # [ 2.759237] usb 1-1: Manufacturer: QEMU gitlab # [ 2.761423] usb 1-1: SerialNumber: 28754-0000:00:01.2-1 gitlab # [ 2.793529] hid: raw HID events driver (C) Jiri Kosina gitlab # [ 2.806511] usbcore: registered new interface driver usbhid gitlab # [ 2.808736] usbhid: USB HID core driver gitlab # [ 2.819213] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 gitlab # [ 2.823301] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0 gitlab # [ 2.933155] systemd[1]: Mounting /sysroot... gitlab # [ 3.232806] EXT4-fs (vda): mounted filesystem 298345a3-4800-42c2-91e8-b535d43c2161 r/w with ordered data mode. Quota mode: none. gitlab # [ 3.092736] systemd[1]: Mounted /sysroot. gitlab # [ 3.094763] systemd[1]: Reached target Initrd Root File System. gitlab # [ 3.097078] systemd[1]: Mounting /sysroot/nix/.ro-store... gitlab # [ 3.099219] systemd[1]: Mounting /sysroot/nix/.rw-store... gitlab # [ 3.101422] systemd[1]: Mounting /sysroot/run... gitlab # [ 3.103535] systemd[1]: Mounting /sysroot/tmp/shared... gitlab # [ 3.105583] systemd[1]: Mounting /sysroot/tmp/xchg... gitlab # [ 3.114562] systemd[1]: Starting Mountpoints Configured in the Real Root... gitlab # [ 3.121386] systemd[1]: Mounted /sysroot/nix/.rw-store. gitlab # [ 3.123590] systemd[1]: Mounted /sysroot/run. gitlab # [ 3.272304] 9p: Installing v9fs 9p2000 file system support gitlab # [ 3.127824] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.132291] systemd[1]: Mounted /sysroot/tmp/shared. gitlab # [ 3.134312] systemd[1]: Mounted /sysroot/tmp/xchg. gitlab # [ 3.140404] systemd[1]: rw-sysroot-nix-store.[ 3.287389] erofs (device vdb): mounted with root inode @ nid 128. gitlab # service: Deactivated successfully. gitlab # [ 3.143366] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.145214] systemd[1]: Mounted /sysroot/nix/.ro-store. gitlab # [ 3.151682] systemd-sysroot-fstab-check[180]: /sysroot should be mounted in the initrd, will request daemon-reload. gitlab # [ 3.154282] systemd[1]: Reload requested from client PID 180 ('systemd-sysroot') (unit initrd-parse-etc.service)... gitlab # [ 3.157085] systemd[1]: Reloading... gitlab # [ 3.211109] systemd[1]: Reloading finished in 57 ms. gitlab # [ 3.238429] systemd-sysroot-fstab-check[180]: Requesting initrd-fs.target/start/replace... gitlab # [ 3.241434] systemd-sysroot-fstab-check[180]: Requesting swap.target/start/replace... gitlab # [ 3.244530] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.246928] systemd[1]: initrd-parse-etc.service: Deactivated successfully. gitlab # [ 3.249490] systemd[1]: Finished Mountpoints Configured in the Real Root. gitlab # [ 3.251950] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. gitlab # [ 3.257851] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.261058] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.933675] systemd[1]: Mounting /sysroot/nix/store... gitlab # [ 3.994517] systemd[1]: Mounted /sysroot/nix/store. gitlab # [ 3.996764] systemd[1]: Reached target Initrd File Systems. gitlab # [ 3.998867] systemd[1]: Starting Find NixOS closure... gitlab # [ 4.001108] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... gitlab # [ 4.030173] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. gitlab # [ 4.032896] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. gitlab # [ 4.051554] systemd[1]: Finished Find NixOS closure. gitlab # [ 4.053479] systemd[1]: Reached target Initrd Default Target. gitlab # [ 4.055502] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... gitlab # [ 4.096347] systemd[1]: Stopped target Initrd Default Target. gitlab # [ 4.098474] systemd[1]: Stopped target Basic System. gitlab # [ 4.100424] systemd[1]: Stopped target Initrd Root Device. gitlab # [ 4.102428] systemd[1]: Stopped target Path Units. gitlab # [ 4.104329] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. gitlab # [ 4.106878] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. gitlab # [ 4.109563] systemd[1]: Stopped target Slice Units. gitlab # [ 4.112288] systemd[1]: Stopped target Socket Units. gitlab # [ 4.115479] systemd[1]: Stopped target System Initialization. gitlab # [ 4.118226] systemd[1]: Stopped target Swaps. gitlab # [ 4.120491] systemd[1]: Stopped target Timer Units. gitlab # [ 4.122523] systemd[1]: dbus.socket: Deactivated successfully. gitlab # [ 4.124783] systemd[1]: Closed D-Bus System Message Bus Socket. gitlab # [ 4.127329] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. gitlab # [ 4.130143] systemd[1]: Stopped Find NixOS closure. gitlab # [ 4.132205] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 4.136156] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 4.138288] systemd[1]: systemd-sysctl.service: Deactivated successfully. gitlab # [ 4.140612] systemd[1]: Stopped Apply Kernel Variables. gitlab # [ 4.142452] systemd[1]: systemd-modules-load.service: Deactivated successfully. gitlab # [ 4.144343] systemd[1]: Stopped Load Kernel Modules. gitlab # [ 4.145742] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. gitlab # [ 4.147934] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. gitlab # [ 4.149902] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. gitlab # [ 4.151807] systemd[1]: Stopped Create System Files and Directories. gitlab # [ 4.153457] systemd[1]: Stopped target Local File Systems. gitlab # [ 4.154906] systemd[1]: Stopped target Preparation for Local File Systems. gitlab # [ 4.156631] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. gitlab # [ 4.158498] systemd[1]: Stopped Coldplug All udev Devices. gitlab # [ 4.159914] systemd[1]: Stopping Rule-based Manager for Device Events and Files... gitlab # [ 4.161505] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 4.162585] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 4.163335] systemd[1]: initrd-cleanup.service: Deactivated successfully. gitlab # [ 4.164330] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. gitlab # [ 4.165211] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 4.166236] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 4.167099] systemd[1]: systemd-udevd.service: Deactivated successfully. gitlab # [ 4.167988] systemd[1]: Stopped Rule-based Manager for Device Events and Files. gitlab # [ 4.168963] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. gitlab # [ 4.170036] systemd[1]: Closed udev Control Socket. gitlab # [ 4.170766] systemd[1]: Starting Cleanup udev Database... gitlab # [ 4.171642] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. gitlab # [ 4.172900] systemd[1]: Stopped Create Static Device Nodes in /dev. gitlab # [ 4.173748] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. gitlab # [ 4.174882] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. gitlab # [ 4.175927] systemd[1]: kmod-static-nodes.service: Deactivated successfully. gitlab # [ 4.176845] systemd[1]: Stopped Create List of Static Device Nodes. gitlab # [ 4.193985] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. gitlab # [ 4.195112] systemd[1]: Finished Cleanup udev Database. gitlab # [ 4.196203] systemd[1]: Reached target Switch Root. gitlab # [ 4.198061] systemd[1]: Starting NixOS Activation... gitlab # [ 4.330974] initrd-nixos-activation-start[228]: booting system configuration /nix/store/fa7yp7hwl9zp8x7hiqk233hjsnpm2rr4-nixos-system-gitlab-test gitlab # [ 4.375225] initrd-nixos-activation-start[228]: running activation script... gitlab # [ 4.618660] initrd-nixos-activation-start[251]: setting up /etc... gitlab # [ 4.705874] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. gitlab # [ 4.707513] systemd[1]: Finished NixOS Activation. gitlab # [ 4.709997] systemd[1]: Starting Switch Root... gitlab # [ 4.745142] systemd[1]: Switching root. gitlab # [ 4.963403] systemd-journald[93]: Received SIGTERM from PID 1 (systemd). gitlab # [ 5.104767] NET: Registered PF_VSOCK protocol family gitlab # [ 5.486292] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 5.496462] systemd[1]: Detected virtualization kvm. gitlab # [ 5.498416] systemd[1]: Detected architecture x86-64. gitlab # [ 5.500492] systemd[1]: Detected first boot. gitlab # [ 5.503124] systemd[1]: Initializing machine ID from random generator. gitlab # [ 5.652492] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 5.749441] systemd[1]: Applying preset policy. gitlab # [ 5.900011] systemd[1]: Populated /etc with preset unit settings. gitlab # [ 6.021936] systemd[1]: initrd-switch-root.service: Deactivated successfully. gitlab # [ 6.023278] systemd[1]: Stopped initrd-switch-root.service. gitlab # [ 6.026252] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. gitlab # [ 6.028457] systemd[1]: Created slice Slice /system/getty. gitlab # [ 6.029999] systemd[1]: Created slice GitLab DevOps Platform Slice. gitlab # [ 6.031483] systemd[1]: Created slice User and Session Slice. gitlab # [ 6.032466] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 6.033865] systemd[1]: Started Forward Password Requests to Wall Directory Watch. gitlab # [ 6.035037] systemd[1]: Expecting device /dev/hvc0... gitlab # [ 6.035804] systemd[1]: Expecting device /dev/ttyS0... gitlab # [ 6.036568] systemd[1]: Reached target Local Encrypted Volumes. gitlab # [ 6.037470] systemd[1]: Stopped target initrd-fs.target. gitlab # [ 6.038238] systemd[1]: Stopped target initrd-root-fs.target. gitlab # [ 6.039081] systemd[1]: Stopped target initrd-switch-root.target. gitlab # [ 6.040006] systemd[1]: Reached target Virtual Machines and Containers. gitlab # [ 6.041072] systemd[1]: Reached target Path Units. gitlab # [ 6.041877] systemd[1]: Reached target Remote File Systems. gitlab # [ 6.042710] systemd[1]: Reached target Slice Units. gitlab # [ 6.043425] systemd[1]: Reached target Swaps. gitlab # [ 6.045014] systemd[1]: Listening on Query the User Interactively for a Password. gitlab # [ 6.047106] systemd[1]: Listening on Process Core Dump Socket. gitlab # [ 6.048509] systemd[1]: Listening on Credential Encryption/Decryption. gitlab # [ 6.050219] systemd[1]: Listening on Factory Reset Management. gitlab # [ 6.051220] systemd[1]: Listening on Hostname Service Socket. gitlab # [ 6.053779] systemd[1]: Starting Journal Log Access Socket... gitlab # [ 6.055293] systemd[1]: Listening on Journal Audit Socket. gitlab # [ 6.057309] systemd[1]: Listening on Console Output Muting Service Socket. gitlab # [ 6.058589] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. gitlab # [ 6.059795] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.061233] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 6.064218] systemd[1]: Listening on Disk Repartitioning Service Socket. gitlab # [ 6.065444] systemd[1]: Listening on udev Control Socket. gitlab # [ 6.066461] systemd[1]: Listening on udev Varlink Socket. gitlab # [ 6.069552] systemd[1]: Mounting Huge Pages File System... gitlab # [ 6.071817] systemd[1]: Mounting POSIX Message Queue File System... gitlab # [ 6.073852] systemd[1]: Mounting Kernel Debug File System... gitlab # [ 6.077735] systemd[1]: Mounting Kernel Trace File System... gitlab # [ 6.090100] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 6.091614] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 6.095880] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 6.098282] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm gitlab # [ 6.103744] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 6.107983] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 6.142806] systemd[1]: Mounting FUSE Control File System... gitlab # [ 6.144596] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 6.152383] systemd[1]: Starting Journal Service... gitlab # [ 6.154443] systemd[1]: Starting Load Kernel Modules... gitlab # [ 6.160361] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... gitlab # [ 6.162570] systemd[1]: Starting Remount Root and Kernel File Systems... gitlab # [ 6.163669] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.168072] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 6.171398] systemd[1]: Listening on Journal Log Access Socket. gitlab # [ 6.172533] systemd[1]: Mounted Huge Pages File System. gitlab # [ 6.173714] systemd[1]: Mounted POSIX Message Queue File System. gitlab # [ 6.175141] systemd[1]: Mounted Kernel Debug File System. gitlab # [ 6.176144] systemd[1]: Mounted Kernel Trace File System. gitlab # [ 6.178987] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 6.180240] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 6.182363] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 6.183582] systemd[1]: Mounted FUSE Control File System. gitlab # [ 6.186871] loop: module loaded gitlab # [ 6.186968] systemd-journald[322]: Collecting audit messages is enabled. gitlab # [ 6.188759] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 6.189790] EXT4-fs (vda): re-mounted 298345a3-4800-42c2-91e8-b535d43c2161. gitlab # [ 6.191763] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 6.050224] systemd[1]: Queued start job for default target Multi-User System. gitlab # [ 6.051695] systemd[1]: Unnecessary job was removed for /dev/vdb. gitlab # [ 6.052960] systemd[1]: systemd-journald.serv[ 6.199723] systemd[1]: Finished Load Kernel Modules. gitlab # ice: Deactivated successfully. gitlab # [ 6.054908] systemd-modules-load[323]: Using 3 probe threads[ 6.201796] systemd[1]: Started Journal Service. gitlab # gitlab # [ 6.056597] systemd-modules-load[323]: Inserted module 'tls' gitlab # [ 6.057835] systemd-modules-load[323]: Inserted module 'loop' gitlab # [ 6.059966] systemd[1]: Finished Remount Root and Kernel File Systems. gitlab # [ 6.062468] systemd[1]: Listening on Disk Image Download Service Socket. gitlab # [ 6.064630] systemd[1]: Starting Firewall... gitlab # [ 6.065989] systemd[1]: Starting Flush Journal to Persistent Storage... gitlab # [ 6.067214] systemd-oomd[325]: No swap; memory pressure usage will be degraded gitlab # [ 6.068696] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 6.070515] systemd[1]: Starting Load/Save OS Random Seed... gitlab # [ 6.072841] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 6.074195] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.075853] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. gitlab # [ 6.076852] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 6.234054] systemd-journald[322]: Received client request to flush runtime journal. gitlab # [ 6.137336] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 6.138835] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 6.140959] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 6.142865] systemd[1]: Finished Load/Save OS Random Seed. gitlab # [ 6.143765] systemd[1]: Reached target First Boot Complete. gitlab # [ 6.144739] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 6.145764] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 6.146871] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 6.148318] systemd-udevd[356]: Using default interface naming scheme 'v261'. gitlab # [ 6.149537] systemd[1]: Finished Flush Journal to Persistent Storage. gitlab # [ 6.170288] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 6.174863] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 6.238313] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 6.265408] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. gitlab # [ 6.274214] (udev-worker)[429]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.299288] (udev-worker)[430]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 6.301489] (udev-worker)[430]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.315942] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. gitlab # [ 6.492275] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console gitlab # [ 6.492353] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 gitlab # [ 6.360343] systemd[1]: Condition check resulted in Virtio network device being skipped. gitlab # [ 6.361867] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 6.366536] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 6.369540] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 6.373122] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.374494] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.498725] ACPI: button: Power Button [PWRF] gitlab # [ 6.499236] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 gitlab # [ 6.499315] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD gitlab # [ 6.529177] Console: switching to colour dummy device 80x25 gitlab # [ 6.532393] [drm] Found bochs VGA, ID 0xb0c5. gitlab # [ 6.532396] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. gitlab # [ 6.534301] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic gitlab # [ 6.535279] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0 gitlab # [ 6.537059] parport_pc 00:03: reported by Plug and Play ACPI gitlab # [ 6.538271] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] gitlab # [ 6.542517] Floppy drive(s): fd0 is 2.88M AMI BIOS gitlab # [ 6.546818] rtc_cmos 00:05: RTC can wake from S4 gitlab # [ 6.555376] FDC 0 is a S82078B gitlab # [ 6.556859] Console: switching to colour frame buffer device 160x50 gitlab # [ 6.556870] rtc_cmos 00:05: registered as rtc0 gitlab # [ 6.556953] rtc_cmos 00:05: setting system clock to 2026-10-04T18:51:23 UTC (1791139883) gitlab # [ 6.560670] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs gitlab # [ 6.561502] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device gitlab # [ 6.576670] mousedev: PS/2 mouse device common for all mice gitlab # [ 6.582860] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 gitlab # [ 6.586653] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 gitlab # [ 6.593595] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input6 gitlab # [ 6.448275] systemd[1]: Finished Firewall. gitlab # [ 6.497490] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.502048] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 6.504313] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 6.508161] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.526084] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 6.527382] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 6.547325] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.715873] ppdev: user-space parallel port driver gitlab # [ 6.737547] kvm_amd: TSC scaling supported gitlab # [ 6.738195] kvm_amd: Nested Virtualization enabled gitlab # [ 6.738796] kvm_amd: Nested Paging enabled gitlab # [ 6.739290] kvm_amd: LBR virtualization supported gitlab # [ 6.739880] kvm_amd: Virtual VMLOAD VMSAVE supported gitlab # [ 6.740448] kvm_amd: Virtual GIF supported gitlab # [ 6.740982] kvm_amd: Virtual NMI enabled gitlab # [ 6.778377] EDAC MC: Ver: 3.0.0 gitlab # [ 6.636759] systemd-vconsole-setup[495]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 6.638461] systemd[1]: Finished Virtual Console Setup. gitlab # [ 6.880386] systemd[1]: Mounting /run/wrappers... gitlab # [ 6.903313] systemd[1]: Mounted /run/wrappers. gitlab # [ 6.905270] systemd[1]: Reached target Local File Systems. gitlab # [ 6.907388] systemd[1]: Listening on Boot Loader Control Service Socket. gitlab # [ 6.909759] systemd[1]: Starting register-nix-paths.service... gitlab # [ 6.911951] systemd[1]: Starting Create SUID/SGID Wrappers... gitlab # [ 6.914156] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 6.917222] systemd[1]: Starting Save Transient machine-id to Disk... gitlab # [ 6.922204] systemd[1]: Starting Create System Files and Directories... gitlab # [ 6.965158] systemd[1]: Finished Create System Files and Directories. gitlab # [ 6.972729] systemd[1]: Starting Rebuild Journal Catalog... gitlab # [ 6.976687] systemd[1]: Starting Record System Boot/Shutdown in UTMP... gitlab # [ 6.981943] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. gitlab # [ 6.986425] systemd[1]: Finished Save Transient machine-id to Disk. gitlab # [ 7.004376] systemd[1]: Finished Record System Boot/Shutdown in UTMP. gitlab # [ 7.021188] systemd[1]: Finished Rebuild Journal Catalog. gitlab # [ 7.023597] systemd[1]: Starting Update is Completed... gitlab # [ 7.039231] systemd[1]: Finished Update is Completed. gitlab # [ 7.077545] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. gitlab # [ 7.078688] systemd[1]: Finished Create SUID/SGID Wrappers. gitlab # [ 7.338290] systemd[1]: Finished register-nix-paths.service. gitlab # [ 7.339386] systemd[1]: Reached target System Initialization. gitlab # [ 7.340312] systemd[1]: Started Discard unused filesystem blocks once a week. gitlab # [ 7.341425] systemd[1]: Started logrotate.timer. gitlab # [ 7.342198] systemd[1]: Started Daily Cleanup of Temporary Directories. gitlab # [ 7.343166] systemd[1]: Reached target Timer Units. gitlab # [ 7.343905] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 7.344813] systemd[1]: Listening on Nix Daemon Socket. gitlab # [ 7.352935] systemd[1]: Starting nixbot CI service socket... gitlab # [ 7.354380] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. gitlab # [ 7.359110] systemd[1]: Starting D-Bus System Message Bus... gitlab # [ 7.360332] systemd[1]: Listening on nixbot CI service socket. gitlab # [ 7.361449] systemd[1]: Reached target Socket Units. gitlab # [ 7.415327] dbus-broker-launch[577]: Looking up NSS user entry for 'systemd-timesync'... gitlab # [ 7.416854] dbus-broker-launch[577]: NSS returned no entry for 'systemd-timesync' gitlab # [ 7.418167] dbus-broker-launch[577]: Invalid user-name in /nix/store/37yal9w16hfdxjh64sww8gn81f4hma6i-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" gitlab # [ 7.423850] systemd[1]: Started D-Bus System Message Bus. gitlab # [ 7.440483] dbus-broker-launch[577]: Ready gitlab # [ 7.443388] systemd[1]: Reached target Basic System. gitlab # [ 7.446449] systemd[1]: Started backdoor.service. gitlab # [ 7.449512] systemd[1]: Starting gitlab-config.service... gitlab # [ 7.452413] systemd[1]: Starting Import lastlog data into lastlog2 database... gitlab # [ 7.460224] systemd[1]: Starting Logrotate configuration check... gitlab # [ 7.465104] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.468993] systemd[1]: Starting Post-Boot Actions... gitlab # [ 7.488347] systemd[1]: Started Reset console on configuration changes. gitlab # connecting to host... gitlab # [ 7.493759] logrotate[582]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. gitlab # [ 7.498116] logrotate[582]: reading config file /nix/store/y3500wq742i80l631mh8nc8rchpsfb9q-logrotate.conf gitlab # [ 7.498685] logrotate[582]: note: 'monthly' overrides previously specified 'weekly' gitlab # [ 7.499487] logrotate[582]: Reading state from file: /var/lib/logrotate.status gitlab # [ 7.500246] logrotate[582]: state file /var/lib/logrotate.status does not exist gitlab # [ 7.500938] logrotate[582]: Allocating hash table for state file, size 64 entries gitlab # [ 7.501676] logrotate[582]: Handling 4 logs gitlab # [ 7.502429] logrotate[582]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.502977] logrotate[582]: considering log /var/log/btmp gitlab # [ 7.503620] logrotate[582]: Creating new state gitlab # [ 7.504143] logrotate[582]: Now: 2026-10-04 18:51 gitlab # [ 7.504600] logrotate[582]: Last rotated at 2026-10-04 18:00 gitlab # [ 7.505189] logrotate[582]: log does not need rotating (log has already been rotated) gitlab # [ 7.505624] logrotate[582]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.506217] logrotate[582]: considering log /var/log/wtmp gitlab # [ 7.506683] logrotate[582]: Creating new state gitlab # [ 7.507218] logrotate[582]: Now: 2026-10-04 18:51 gitlab # [ 7.507657] logrotate[582]: Last rotated at 2026-10-04 18:00 gitlab # [ 7.508192] logrotate[582]: log does not need rotating (log has already been rotated) gitlab # [ 7.508655] logrotate[582]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed gitlab # [ 7.509195] logrotate[582]: switching euid from 0 to 165 and egid from 0 to 165 (pid 582) gitlab # [ 7.509625] logrotate[582]: considering log /var/gitlab/state/log/*.log gitlab # [ 7.510205] logrotate[582]: log /var/gitlab/state/log/*.log does not exist -- skipping gitlab # [ 7.510647] logrotate[582]: Creating new state gitlab # [ 7.511182] logrotate[582]: switching euid from 165 to 0 and egid from 165 to 0 (pid 582) gitlab # [ 7.511624] logrotate[582]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed gitlab # [ 7.512154] logrotate[582]: switching euid from 0 to 60 and egid from 0 to 60 (pid 582) gitlab # [ 7.512605] logrotate[582]: considering log /var/log/nginx/*.log gitlab # [ 7.513162] logrotate[582]: log /var/log/nginx/*.log does not exist -- skipping gitlab # [ 7.513605] logrotate[582]: Creating new state gitlab # [ 7.514186] logrotate[582]: not running postrotate script, since no logs were rotated gitlab # [ 7.514620] logrotate[582]: switching euid from 60 to 0 and egid from 60 to 0 (pid 582) gitlab # [ 7.515184] systemd[1]: Starting resolvconf update... gitlab # [ 7.532198] systemd[1]: Finished Logrotate configuration check. gitlab: Guest shell says: b'Spawning backdoor root shell...\n' gitlab # [ 7.537667] systemd[1]: Finished Post-Boot Actions. gitlab: connected to guest root shell gitlab: (connecting took 8.17 seconds) gitlab: (finished: waiting for the VM to finish booting, in 8.17 seconds) gitlab # [ 7.551721] systemd[1]: Finished Import lastlog data into lastlog2 database. gitlab # [ 7.558072] nsncd[583]: Oct 04 18:51:24.647 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.558739] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.559659] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.560447] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.567155] systemd[1]: Starting User Login Management... gitlab # [ 7.607468] systemd[1]: Stopped target Host and Network Name Lookups. gitlab # [ 7.607667] systemd[1]: Stopping Host and Network Name Lookups... gitlab # [ 7.608072] systemd[1]: Stopped target User and Group Name Lookups. gitlab # [ 7.608710] systemd[1]: Stopping User and Group Name Lookups... gitlab # [ 7.610155] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... gitlab # [ 7.612607] systemd[1]: nscd.service: Deactivated successfully. gitlab # [ 7.613473] systemd-logind[627]: New seat seat0. gitlab # [ 7.613888] systemd[1]: Stopped Name Service Cache Daemon (nsncd). gitlab # [ 7.614214] systemd[1]: nscd.service: Consumed 34ms CPU time over 148ms wall clock time, 7.3M memory peak, 5.5M read from disk. gitlab # [ 7.614638] systemd-logind[627]: Watching system buttons on /dev/input/event2 (Power Button) gitlab # [ 7.618262] systemd-logind[627]: Watching system buttons on /dev/input/event5 (QEMU Virtio Keyboard) gitlab # [ 7.618548] systemd-logind[627]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) gitlab # [ 7.618736] systemd[1]: Started User Login Management. gitlab # [ 7.627952] systemd[1]: Starting linger-users.service... gitlab # [ 7.631576] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.645846] systemd[1]: linger-users.service: Deactivated successfully. gitlab # [ 7.646044] systemd[1]: Finished linger-users.service. gitlab # [ 7.648112] nsncd[674]: Oct 04 18:51:24.737 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.648934] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.649875] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.650545] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.663705] systemd[1]: Finished resolvconf update. gitlab # [ 7.664733] systemd[1]: Reached target Preparation for Network. gitlab # [ 7.667520] systemd[1]: Starting DHCP Client... gitlab # [ 7.668763] systemd[1]: Starting Address configuration of eth1... gitlab # [ 7.669987] systemd[1]: Starting Extra networking commands.... gitlab # [ 7.701400] network-addresses-eth1-start[700]: adding address 192.168.1.1/24... done gitlab # [ 7.707428] network-addresses-eth1-start[700]: adding address 2001:db8:1::1/64... done gitlab # [ 7.712514] systemd[1]: Finished Address configuration of eth1. gitlab # [ 7.744493] dhcpcd[709]: dhcpcd-10.3.2 starting gitlab # [ 7.747419] dhcpcd[764]: dev: loaded udev gitlab # [ 7.747782] systemd[1]: Finished Extra networking commands.. gitlab # [ 7.748294] systemd[1]: Reached target Network. gitlab # [ 7.750736] systemd[1]: Starting gitlab-workhorse.service... gitlab # [ 7.752425] systemd[1]: Starting Nginx Web Server... gitlab # [ 7.903031] 8021q: 802.1Q VLAN Support v1.8 gitlab # [ 7.903583] 8021q: adding VLAN 0 to HW filter on device eth1 gitlab # [ 7.758385] systemd[1]: Starting PostgreSQL Server... gitlab # [ 7.765109] systemd[1]: Starting Redis Server - redis-gitlab... gitlab # [ 7.766093] systemd[1]: Starting Permit User Sessions... gitlab # [ 7.771863] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. gitlab # [ 7.785724] systemd[1]: Finished Permit User Sessions. gitlab # [ 7.793492] systemd[1]: Started Getty on tty1. gitlab # [ 7.794341] systemd[1]: Reached target Login Prompts. gitlab # [ 7.992168] cfg80211: Loading compiled-in X.509 certificates for regulatory database gitlab # [ 7.997927] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' gitlab # [ 7.999306] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' gitlab # [ 8.000768] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 gitlab # [ 8.002253] cfg80211: failed to load regulatory.db gitlab # [ 8.031111] 8021q: adding VLAN 0 to HW filter on device eth0 gitlab # [ 7.885832] dhcpcd[764]: eth0: waiting for carrier gitlab # [ 7.887158] dhcpcd[764]: eth0: carrier acquired gitlab # [ 7.895623] dhcpcd[764]: DUID 00:01:00:01:32:55:5c:ac:52:54:00:12:34:56 gitlab # [ 7.896318] dhcpcd[764]: eth0: IAID 00:12:34:56 gitlab # [ 7.896665] dhcpcd[764]: eth0: adding address fe80::5054:ff:fe12:3456 gitlab # [ 7.907230] nginx-pre-start[793]: nginx: the configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf syntax is ok gitlab # [ 7.907613] nginx-pre-start[793]: nginx: configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf test is successful gitlab # [ 7.911271] systemd[1]: Started Nginx Web Server. gitlab # [ 7.912411] redis-server[788]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. gitlab # [ 7.913233] redis-server[788]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo gitlab # [ 7.914142] redis-server[788]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=788, just started gitlab # [ 7.914699] redis-server[788]: Configuration loaded gitlab # [ 7.915467] redis[788]: monotonic clock: POSIX clock_gettime gitlab # [ 7.916078] redis[788]: Running mode=standalone, port=0. gitlab # [ 7.916565] redis[788]: Server initialized gitlab # [ 7.917145] redis[788]: Ready to accept connections unix gitlab # [ 7.917685] systemd[1]: Started Redis Server - redis-gitlab. gitlab # [ 7.918270] redis[788]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. gitlab # [ 7.941960] postgresql-pre-start[794]: The files belonging to this database system will be owned by user "postgres". gitlab # [ 7.942611] postgresql-pre-start[794]: This user must also own the server process. gitlab # [ 7.946248] postgresql-pre-start[794]: The database cluster will be initialized with locale "en_US.UTF-8". gitlab # [ 7.948950] postgresql-pre-start[794]: The default database encoding has accordingly been set to "UTF8". gitlab # [ 7.950524] postgresql-pre-start[794]: The default text search configuration will be set to "english". gitlab # [ 7.951811] postgresql-pre-start[794]: Data page checksums are enabled. gitlab # [ 7.952796] postgresql-pre-start[794]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok gitlab # [ 7.954192] postgresql-pre-start[794]: creating subdirectories ... ok gitlab # [ 7.955263] postgresql-pre-start[794]: selecting dynamic shared memory implementation ... posix gitlab # [ 7.977418] postgresql-pre-start[794]: selecting default "max_connections" ... 100 gitlab # [ 8.014574] postgresql-pre-start[794]: selecting default "shared_buffers" ... 128MB gitlab # [ 8.112253] dhcpcd[764]: eth0: soliciting a DHCP lease gitlab # [ 8.271251] NET: Registered PF_PACKET protocol family gitlab # [ 8.140064] dhcpcd[764]: eth0: offered 10.0.2.15 from 10.0.2.2 gitlab # [ 8.143373] dhcpcd[764]: eth0: probing address 10.0.2.15/24 gitlab # [ 8.292392] systemd[1]: Started gitlab-workhorse.service. gitlab # [ 8.429275] postgresql-pre-start[794]: selecting default time zone ... UTC gitlab # [ 8.432217] postgresql-pre-start[794]: creating configuration files ... ok gitlab # [ 8.516613] gitlab-workhorse[826]: time="2026-10-04T18:51:25Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4 gitlab # [ 8.516854] gitlab-workhorse[826]: time="2026-10-04T18:51:25Z" level=info msg="Using redis/go-redis" gitlab # [ 8.517499] gitlab-workhorse[826]: time="2026-10-04T18:51:25Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix gitlab # [ 8.614136] postgresql-pre-start[794]: running bootstrap script ... ok gitlab # [ 9.035649] postgresql-pre-start[794]: performing post-bootstrap initialization ... ok gitlab # [ 9.309732] postgresql-pre-start[794]: syncing data to disk ... ok gitlab # [ 9.310194] postgresql-pre-start[794]: initdb: warning: enabling "trust" authentication for local connections gitlab # [ 9.311288] postgresql-pre-start[794]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. gitlab # [ 9.312449] postgresql-pre-start[794]: Success. You can now start the database server using: gitlab # [ 9.313960] postgresql-pre-start[794]: pg_ctl -D /var/lib/postgresql/18 -l logfile start gitlab # [ 9.439417] postgres[850]: [850] LOG: starting PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit gitlab # [ 9.440293] postgres[850]: [850] LOG: listening on IPv6 address "::1", port 5432 gitlab # [ 9.441073] postgres[850]: [850] LOG: listening on IPv4 address "127.0.0.1", port 5432 gitlab # [ 9.448815] postgres[850]: [850] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" gitlab # [ 9.461471] postgres[860]: [860] LOG: database system was shut down at 2026-10-04 18:51:26 GMT gitlab # [ 9.468869] postgres[850]: [850] LOG: database system is ready to accept connections gitlab # [ 9.479451] systemd[1]: Started PostgreSQL Server. gitlab # [ 9.483469] systemd[1]: Starting PostgreSQL Setup Scripts... gitlab # [ 9.655468] postgresql-setup-start[875]: CREATE DATABASE gitlab # [ 9.688667] postgresql-setup-start[890]: CREATE ROLE gitlab # [ 9.701408] postgresql-setup-start[892]: ALTER DATABASE gitlab # [ 9.703403] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[874]: mkdir -p /var/gitlab/state/home/.ssh: OK gitlab # [ 9.705889] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[874]: chmod 700 /var/gitlab/state/home/.ssh: OK gitlab # [ 9.722271] postgresql-setup-start[901]: CREATE ROLE gitlab # [ 9.724713] systemd[1]: Finished PostgreSQL Setup Scripts. gitlab # [ 9.726407] systemd[1]: Reached target PostgreSQL. gitlab # [ 9.727849] systemd[1]: Starting gitlab-postgresql.service... gitlab # [ 9.748497] systemd[1]: Finished gitlab-config.service. gitlab # [ 9.751585] systemd[1]: Started gitaly.service. gitlab # [ 9.753192] systemd[1]: Starting gitlab-db-config.service... gitlab # [ 9.826899] gitlab-postgresql-start[919]: CREATE DATABASE gitlab # [ 9.872208] gitlab-postgresql-start[926]: CREATE EXTENSION gitlab # [ 9.938451] dhcpcd[764]: eth0: soliciting an IPv6 router gitlab # [ 9.939858] dhcpcd[764]: eth0: Router Advertisement from fe80::2 gitlab # [ 9.941450] dhcpcd[764]: eth0: adding address fec0::5054:ff:fe12:3456/64 gitlab # [ 9.942464] dhcpcd[764]: eth0: adding route to fec0::/64 gitlab # [ 9.943443] dhcpcd[764]: eth0: adding default route via fe80::2 gitlab # [ 9.960404] gitlab-postgresql-start[928]: CREATE EXTENSION gitlab # [ 9.966279] systemd[1]: Finished gitlab-postgresql.service. gitlab # [ 10.349168] gitaly[916]: time="2026-10-04T18:51:27.437Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/ymnlr20kjfmhkzg7i9hp25h3ii8qnn0d-gitaly.toml' and fix the errors that are printed." pid=916 gitlab # [ 10.349946] gitaly[916]: time="2026-10-04T18:51:27.438Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=916 gitlab # [ 10.351413] gitaly[916]: time="2026-10-04T18:51:27.438Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=916 gitlab # [ 10.351934] gitaly[916]: time="2026-10-04T18:51:27.438Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=916 gitlab # [ 10.353207] gitaly[916]: time="2026-10-04T18:51:27.438Z" level=info msg="Starting Gitaly" pid=916 version= gitlab # [ 10.575939] gitaly[916]: time="2026-10-04T18:51:27.665Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=211 pid=916 gitlab # [ 10.576330] gitaly[916]: time="2026-10-04T18:51:27.665Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=916 gitlab # [ 10.576428] gitaly[916]: time="2026-10-04T18:51:27.666Z" level=info msg="finished initializing command factory" duration_ms=0 pid=916 gitlab # [ 10.576971] gitaly[916]: time="2026-10-04T18:51:27.666Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-916/git-exec-3975435044.d/git pid=916 gitlab # [ 10.583176] gitaly[916]: time="2026-10-04T18:51:27.672Z" level=info msg="finished detecting git version" duration_ms=6 pid=916 gitlab # [ 10.583446] gitaly[916]: time="2026-10-04T18:51:27.672Z" level=info msg="using Git version" pid=916 version=2.53.g gitlab # [ 10.583987] gitaly[916]: time="2026-10-04T18:51:27.673Z" level=info msg="clearing disk cache object folder" pid=916 storage=default gitlab # [ 10.584628] gitaly[916]: time="2026-10-04T18:51:27.673Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2199758701 pid=916 storage=default gitlab # [ 10.585174] gitaly[916]: time="2026-10-04T18:51:27.673Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2199758701 pid=916 storage=default gitlab # [ 10.585408] gitaly[916]: time="2026-10-04T18:51:27.673Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=916 gitlab # [ 10.585988] gitaly[916]: time="2026-10-04T18:51:27.673Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=916 gitlab # [ 10.586752] gitaly[916]: time="2026-10-04T18:51:27.673Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2199758701 clear_duration_ms=0 pid=916 storage=default gitlab # [ 10.629715] gitaly[916]: time="2026-10-04T18:51:27.718Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=916 gitlab # [ 10.630407] gitaly[916]: time="2026-10-04T18:51:27.718Z" level=info msg="counting repositories" pid=916 gitlab # [ 10.631371] gitaly[916]: time="2026-10-04T18:51:27.719Z" level=info msg="starting to count repositories" pid=916 storage_path=/var/gitlab/state/repositories gitlab # [ 10.632363] gitaly[916]: time="2026-10-04T18:51:27.719Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=916 storage_path=/var/gitlab/state/repositories gitlab # [ 10.633950] gitaly[916]: time="2026-10-04T18:51:27.719Z" level=info msg="completed counting all repositories" counting_duration=0 pid=916 gitlab # [ 10.634954] gitaly[916]: time="2026-10-04T18:51:27.718Z" level=info msg="finished tempdir cleaner walk" pid=916 storage=default time_ms=0 gitlab # [ 10.635781] gitaly[916]: time="2026-10-04T18:51:27.718Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=916 schema=unix gitlab # [ 10.636612] gitaly[916]: time="2026-10-04T18:51:27.719Z" level=info msg="listening at address" address=/run/gitaly/gitaly-916/sock.d/intern pid=916 schema=unix gitlab # [ 10.645770] gitaly[916]: time="2026-10-04T18:51:27.735Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=916 gitlab # [ 10.646573] gitaly[916]: time="2026-10-04T18:51:27.735Z" level=info msg="Started Gitaly" duration_ms=296 pid=916 gitlab # [ 10.647314] gitaly[916]: time="2026-10-04T18:51:27.735Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=916 gitlab # [ 10.647835] gitaly[916]: time="2026-10-04T18:51:27.736Z" level=info msg="maintenance: daily scheduled" pid=916 scheduled="2026-10-05 12:00:00 +0000 UTC" gitlab # [ 11.533213] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[922]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 11.535906] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[922]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 11.572783] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[922]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 12.645441] gitaly[916]: time="2026-10-04T18:51:29.733Z" level=info msg="License database preloaded" duration_ms=2290 pid=916 gitlab # [ 13.698193] dhcpcd[764]: eth0: leased 10.0.2.15 for 86400 seconds gitlab # [ 13.698997] dhcpcd[764]: eth0: adding route to 10.0.2.0/24 gitlab # [ 13.700470] dhcpcd[764]: eth0: adding default route via 10.0.2.2 gitlab # [ 13.840891] systemd[1]: Started DHCP Client. gitlab # [ 13.844583] systemd[1]: Reached target Network is Online. gitlab # [ 43.003608] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[922]: W, [2026-10-04T18:52:00.092243 #922] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 45.091229] postgres[1326]: [1326] ERROR: relation "feature_gates" does not exist at character 523 gitlab # [ 45.091641] postgres[1326]: [1326] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), gitlab # [ 45.093467] postgres[1326]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, gitlab # [ 45.094993] postgres[1326]: c.collname, col_description(a.attrelid, a.attnum) AS comment, gitlab # [ 45.097067] postgres[1326]: attidentity AS identity, gitlab # [ 45.098960] postgres[1326]: attgenerated as attgenerated gitlab # [ 45.101377] postgres[1326]: FROM pg_attribute a gitlab # [ 45.102898] postgres[1326]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum gitlab # [ 45.105296] postgres[1326]: LEFT JOIN pg_type t ON a.atttypid = t.oid gitlab # [ 45.106434] postgres[1326]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation gitlab # [ 45.107943] postgres[1326]: WHERE a.attrelid = '"feature_gates"'::regclass gitlab # [ 45.110307] postgres[1326]: AND a.attnum > 0 AND NOT a.attisdropped gitlab # [ 45.112242] postgres[1326]: ORDER BY a.attnum gitlab # [ 45.113075] postgres[1326]: gitlab # [ 48.565857] gitlab-workhorse[826]: time="2026-10-04T18:52:05Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 48.611094] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1359]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping gitlab # [ 48.611921] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1359]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping gitlab # [ 58.567281] gitlab-workhorse[826]: time="2026-10-04T18:52:15Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 64.361534] postgres[1434]: [1434] ERROR: relation "table_without_models" does not exist gitlab # [ 64.362528] postgres[1434]: [1434] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 68.576068] gitlab-workhorse[826]: time="2026-10-04T18:52:25Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 76.813431] systemd[1]: Finished gitlab-db-config.service. gitlab # [ 76.817510] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 76.820760] systemd[1]: Starting gitlab.service... gitlab # [ 77.198561] bundle[1610]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 77.199376] bundle[1610]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 77.200277] bundle[1610]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 78.586248] gitlab-workhorse[826]: time="2026-10-04T18:52:35Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 79.979428] sidekiq-cluster[1609]: {"severity":"INFO","time":"2026-10-04T18:52:37.051Z","message":"Starting cluster with 1 processes"} gitlab # [ 80.085142] bundle[1610]: {"timestamp":"2026-10-04T18:52:37.153Z","pid":1610,"message":"* Enabling systemd notification integration"} gitlab # [ 80.085845] bundle[1610]: {"timestamp":"2026-10-04T18:52:37.173Z","pid":1610,"message":"Puma starting in cluster mode..."} gitlab # [ 80.086817] bundle[1610]: {"timestamp":"2026-10-04T18:52:37.173Z","pid":1610,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"} gitlab # [ 80.087691] bundle[1610]: {"timestamp":"2026-10-04T18:52:37.173Z","pid":1610,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 80.088626] bundle[1610]: {"timestamp":"2026-10-04T18:52:37.173Z","pid":1610,"message":"* Min threads: 0"} gitlab # [ 80.089484] bundle[1610]: {"timestamp":"2026-10-04T18:52:37.173Z","pid":1610,"message":"* Max threads: 4"} gitlab # [ 80.090290] bundle[1610]: {"timestamp":"2026-10-04T18:52:37.173Z","pid":1610,"message":"* Environment: production"} gitlab # [ 80.090944] bundle[1610]: {"timestamp":"2026-10-04T18:52:37.174Z","pid":1610,"message":"* Master PID: 1610"} gitlab # [ 80.091671] bundle[1610]: {"timestamp":"2026-10-04T18:52:37.174Z","pid":1610,"message":"* Workers: 2"} gitlab # [ 80.092548] bundle[1610]: {"timestamp":"2026-10-04T18:52:37.174Z","pid":1610,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} gitlab # [ 80.093287] bundle[1610]: {"timestamp":"2026-10-04T18:52:37.174Z","pid":1610,"message":"* Preloading application"} gitlab # [ 88.595901] gitlab-workhorse[826]: time="2026-10-04T18:52:45Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 98.597648] gitlab-workhorse[826]: time="2026-10-04T18:52:55Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 103.455396] bundle[1610]: W, [2026-10-04T18:53:00.544108 #1610] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 105.669991] sidekiq-cluster[1640]: 2026-10-04T18:53:02.758Z pid=1640 tid=wjk INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1640, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>450560000} gitlab # [ 105.844945] sidekiq-cluster[1640]: W, [2026-10-04T18:53:02.934261 #1640] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 105.888548] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:02.977Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 108.053148] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.141Z","message":"GitLab reliable fetch activated!"} gitlab # [ 108.057237] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.141Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 108.077146] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.166Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} gitlab # [ 108.082471] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.169Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"} gitlab # [ 108.087972] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.171Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} gitlab # [ 108.089166] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.174Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} gitlab # [ 108.090258] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.176Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} gitlab # [ 108.106701] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.196Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} gitlab # [ 108.111976] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.200Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} gitlab # [ 108.119120] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.208Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} gitlab # [ 108.125816] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.215Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} gitlab # [ 108.132737] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.222Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} gitlab # [ 108.139974] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.229Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} gitlab # [ 108.146905] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.236Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} gitlab # [ 108.152528] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.242Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} gitlab # [ 108.158597] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.248Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} gitlab # [ 108.167075] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.256Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} gitlab # [ 108.170894] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.260Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} gitlab # [ 108.175694] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.265Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} gitlab # [ 108.180259] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.269Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} gitlab # [ 108.184683] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.274Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} gitlab # [ 108.188924] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.278Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} gitlab # [ 108.193351] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.282Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} gitlab # [ 108.197508] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.287Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} gitlab # [ 108.201566] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.291Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} gitlab # [ 108.205478] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.295Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} gitlab # [ 108.209357] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.298Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} gitlab # [ 108.213930] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.303Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} gitlab # [ 108.216897] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.306Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} gitlab # [ 108.219983] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.309Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} gitlab # [ 108.222905] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.312Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} gitlab # [ 108.225941] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.315Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} gitlab # [ 108.228331] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.317Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} gitlab # [ 108.230873] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.320Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} gitlab # [ 108.233386] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.323Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} gitlab # [ 108.235607] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.325Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} gitlab # [ 108.237615] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.327Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} gitlab # [ 108.239431] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.329Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} gitlab # [ 108.241682] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.331Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} gitlab # [ 108.243533] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.333Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} gitlab # [ 108.245313] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.334Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} gitlab # [ 108.247509] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.337Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} gitlab # [ 108.249543] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.339Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} gitlab # [ 108.251725] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.341Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} gitlab # [ 108.254269] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.343Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} gitlab # [ 108.256536] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.346Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} gitlab # [ 108.258778] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.348Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} gitlab # [ 108.260409] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.350Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} gitlab # [ 108.262596] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.352Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} gitlab # [ 108.265314] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.354Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} gitlab # [ 108.294941] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.384Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} gitlab # [ 108.296915] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.386Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} gitlab # [ 108.299324] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.388Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} gitlab # [ 108.302552] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.392Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} gitlab # [ 108.311644] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.401Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} gitlab # [ 108.316931] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.406Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} gitlab # [ 108.324001] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.413Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} gitlab # [ 108.329280] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.418Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} gitlab # [ 108.335507] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.425Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} gitlab # [ 108.341164] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.430Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} gitlab # [ 108.346602] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.436Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} gitlab # [ 108.351828] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.441Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} gitlab # [ 108.357180] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.446Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} gitlab # [ 108.361386] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.450Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} gitlab # [ 108.367664] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.457Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} gitlab # [ 108.373642] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.463Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} gitlab # [ 108.376726] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.466Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} gitlab # [ 108.381536] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.471Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} gitlab # [ 108.385106] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.474Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} gitlab # [ 108.389396] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.478Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} gitlab # [ 108.393088] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.482Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} gitlab # [ 108.396628] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.486Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} gitlab # [ 108.400591] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.490Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} gitlab # [ 108.404929] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.494Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} gitlab # [ 108.409427] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.499Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} gitlab # [ 108.413263] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.502Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} gitlab # [ 108.416759] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.506Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} gitlab # [ 108.421488] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.511Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} gitlab # [ 108.425724] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.515Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} gitlab # [ 108.430679] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.520Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} gitlab # [ 108.434771] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.524Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} gitlab # [ 108.439357] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.528Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} gitlab # [ 108.443213] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.532Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} gitlab # [ 108.448219] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.537Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} gitlab # [ 108.452536] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.542Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} gitlab # [ 108.456201] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.545Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} gitlab # [ 108.459908] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.549Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} gitlab # [ 108.463390] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.552Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} gitlab # [ 108.468380] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.557Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} gitlab # [ 108.472125] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.561Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} gitlab # [ 108.476092] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.565Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} gitlab # [ 108.479675] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.569Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} gitlab # [ 108.483615] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.573Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} gitlab # [ 108.487635] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.577Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} gitlab # [ 108.491866] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.581Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} gitlab # [ 108.496562] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.586Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} gitlab # [ 108.501113] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.590Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} gitlab # [ 108.506475] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.596Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} gitlab # [ 108.510407] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.599Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} gitlab # [ 108.517140] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.606Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} gitlab # [ 108.535841] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.625Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} gitlab # [ 108.538685] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.628Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} gitlab # [ 108.542537] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.632Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} gitlab # [ 108.547378] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.636Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} gitlab # [ 108.551808] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.641Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} gitlab # [ 108.555881] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.645Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} gitlab # [ 108.563367] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.652Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} gitlab # [ 108.566267] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.655Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} gitlab # [ 108.568580] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.657Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} gitlab # [ 108.571382] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.660Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} gitlab # [ 108.573614] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:05.662Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} gitlab # [ 108.606925] gitlab-workhorse[826]: time="2026-10-04T18:53:05Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 118.616703] gitlab-workhorse[826]: time="2026-10-04T18:53:15Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 128.617475] gitlab-workhorse[826]: time="2026-10-04T18:53:25Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 130.713082] postgres[2105]: [2105] ERROR: relation "table_without_models" does not exist gitlab # [ 130.713296] postgres[2105]: [2105] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 131.358415] postgres[2126]: [2126] ERROR: relation "table_without_models" does not exist gitlab # [ 131.361392] postgres[2126]: [2126] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 131.717902] sidekiq-cluster[1640]: InitializerConnections Query: SET client_min_messages TO 'warning' gitlab # [ 131.721069] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 131.724434] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 131.727440] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 131.730622] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 131.733453] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?' gitlab # [ 131.736320] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start' gitlab # [ 131.739247] sidekiq-cluster[1640]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 131.742288] sidekiq-cluster[1640]: InitializerConnections Query: SET standard_conforming_strings = on gitlab # [ 131.744533] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 131.747221] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 131.749539] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 131.752076] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 131.754333] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?' gitlab # [ 131.755756] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start' gitlab # [ 131.757269] sidekiq-cluster[1640]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 131.772665] sidekiq-cluster[1640]: InitializerConnections Query: SET intervalstyle = iso_8601 gitlab # [ 131.773972] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 131.776741] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 131.776898] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 131.778109] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 131.778989] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?' gitlab # [ 131.780206] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start' gitlab # [ 131.781084] sidekiq-cluster[1640]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 131.782024] sidekiq-cluster[1640]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT t.oid, t.typname gitlab # [ 131.782864] sidekiq-cluster[1640]: FROM pg_type as t gitlab # [ 131.784033] sidekiq-cluster[1640]: WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'numeric', 'bool', 'timestamp', 'timestamptz') gitlab # [ 131.784864] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 131.785538] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 131.786046] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 131.786545] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 131.787047] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?' gitlab # [ 131.787468] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start' gitlab # [ 131.787965] sidekiq-cluster[1640]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 131.801575] sidekiq-cluster[1640]: InitializerConnections Query: SET SESSION timezone TO 'UTC' gitlab # [ 131.801848] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 131.802468] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 131.802982] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 131.803630] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 131.804181] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?' gitlab # [ 131.804616] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start' gitlab # [ 131.805148] sidekiq-cluster[1640]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 131.843673] sidekiq-cluster[1640]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT "feature_gates"."key", "feature_gates"."value" FROM "feature_gates" WHERE "feature_gates"."feature_key" = 'sidekiq_throttling_middleware' gitlab # [ 131.847777] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 131.847891] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 131.849098] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 131.850031] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 131.850847] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?' gitlab # [ 131.851890] sidekiq-cluster[1640]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start' gitlab # [ 131.853165] sidekiq-cluster[1640]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 133.283198] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:30.372Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 133.286498] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:30.375Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 133.288251] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:30.376Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 133.289661] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:30.376Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 133.291735] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:30.377Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 134.211903] bundle[1610]: {"timestamp":"2026-10-04T18:53:31.300Z","pid":1610,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} gitlab # [ 134.216446] bundle[1610]: {"timestamp":"2026-10-04T18:53:31.300Z","pid":1610,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} gitlab # [ 134.220381] bundle[1610]: {"timestamp":"2026-10-04T18:53:31.300Z","pid":1610,"message":"! #\u003cThread:0x000074e0d16acbe8 /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} gitlab # [ 134.231830] bundle[1610]: {"timestamp":"2026-10-04T18:53:31.300Z","pid":1610,"message":"! #\u003cThread:0x000074e0d18afe18 /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} gitlab # [ 134.243176] bundle[1610]: {"timestamp":"2026-10-04T18:53:31.300Z","pid":1610,"message":"Use Ctrl-C to stop"} gitlab # [ 134.246277] bundle[1610]: {"timestamp":"2026-10-04T18:53:31.301Z","pid":1610,"message":"Sending status to systemd every 1.0 sec"} gitlab # [ 134.851593] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:31.940Z","message":"Cleaning working queues"} gitlab # [ 134.855404] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:31.941Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"e84c761d9202c0257cbdf48a","created_at":"2026-10-04T18:52:33.552Z","trace_propagation_headers":{"sentry-trace":"bee31fd0f56445e6b78879388948ff67-72e38558f80041e7","baggage":"sentry-trace_id=bee31fd0f56445e6b78879388948ff67,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"089ba6dd9cdd4c583d36016a6b7fc864","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-04T18:52:33.560Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-e84c761d9202c0257cbdf48a: start","job_status":"start","queue_duration_s":58.380556,"scheduling_latency_s":58.3806} gitlab # [ 134.936182] bundle[1610]: {"timestamp":"2026-10-04T18:53:32.025Z","pid":1610,"message":"- Worker 0 (PID: 2174) booted in 0.69s, phase: 0"} gitlab # [ 134.967102] bundle[1610]: {"timestamp":"2026-10-04T18:53:32.056Z","pid":1610,"message":"- Worker 1 (PID: 2178) booted in 0.69s, phase: 0"} gitlab # [ 134.972089] systemd[1]: Started gitlab.service. gitlab # [ 134.973944] systemd[1]: Reached target Common target for all GitLab services.. gitlab # [ 134.976404] systemd[1]: Starting setup-gitlab.service... gitlab # [ 134.990840] setup-gitlab-start[2218]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' gitlab # [ 135.002174] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:53:32.091Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"e84c761d9202c0257cbdf48a","created_at":"2026-10-04T18:52:33.552Z","trace_propagation_headers":{"sentry-trace":"bee31fd0f56445e6b78879388948ff67-72e38558f80041e7","baggage":"sentry-trace_id=bee31fd0f56445e6b78879388948ff67,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"089ba6dd9cdd4c583d36016a6b7fc864","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-04T18:52:33.560Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-e84c761d9202c0257cbdf48a: done: 0.150185 sec","job_status":"done","queue_duration_s":58.380556,"scheduling_latency_s":58.3806,"redis_calls":24,"redis_duration_s":0.008347,"redis_read_bytes":912,"redis_write_bytes":2622,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.002262,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.005088,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000997,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057612,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.073384745,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.150185,"completed_at":"2026-10-04T18:53:32.091Z","load_balancing_strategy":"primary","db_duration_s":0.039323,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 137.165620] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:53:34 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 2100 gitlab # [ 137.172329] setup-gitlab-start[2218]: + TOKEN=nixbot-test-token-of-20-chars gitlab # [ 137.173156] setup-gitlab-start[2218]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n' gitlab # [ 137.194578] runuser[2236]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) gitlab # [ 137.585269] setup-gitlab-start[2237]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 137.585750] setup-gitlab-start[2237]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 137.587337] setup-gitlab-start[2237]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 138.163514] setup-gitlab-start[2237]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 138.164168] setup-gitlab-start[2237]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 138.165453] setup-gitlab-start[2237]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 163.710449] setup-gitlab-start[2237]: W, [2026-10-04T18:54:00.799182 #2237] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 170.618327] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:07.706Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"699fc1c46920052ea87ba6e4","created_at":"2026-10-04T18:54:07.631Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"906fa33240d9b421b56067ee084e1441","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:07.704Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-699fc1c46920052ea87ba6e4: start","job_status":"start","queue_duration_s":0.002481,"scheduling_latency_s":0.002521} gitlab # [ 170.750701] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:07.839Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"699fc1c46920052ea87ba6e4","created_at":"2026-10-04T18:54:07.631Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"906fa33240d9b421b56067ee084e1441","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:07.704Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-699fc1c46920052ea87ba6e4: done: 0.133085 sec","job_status":"done","queue_duration_s":0.002481,"scheduling_latency_s":0.002521,"redis_calls":12,"redis_duration_s":0.012409,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.005554,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003981,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002874,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001716978,"exclusive_lock_hold_duration_s":0.019355812,"duration_s":0.133085,"completed_at":"2026-10-04T18:54:07.839Z","load_balancing_strategy":"primary","db_duration_s":0.031891,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 170.755745] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:07.845Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"b8b4d40789d71d5ad3bd9271","created_at":"2026-10-04T18:54:07.713Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:07.725Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-b8b4d40789d71d5ad3bd9271: start","job_status":"start","queue_duration_s":0.119874,"scheduling_latency_s":0.119903} gitlab # [ 170.925432] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.014Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"b8b4d40789d71d5ad3bd9271","created_at":"2026-10-04T18:54:07.713Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:07.725Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-b8b4d40789d71d5ad3bd9271: done: 0.169498 sec","job_status":"done","queue_duration_s":0.119874,"scheduling_latency_s":0.119903,"redis_calls":71,"redis_duration_s":0.059173,"redis_read_bytes":120,"redis_write_bytes":42624,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.00628,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001583,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.047902,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.003408,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.07635,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000569067,"duration_s":0.169498,"completed_at":"2026-10-04T18:54:08.014Z","load_balancing_strategy":"primary","db_duration_s":0.003951,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 170.927147] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.016Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"8e9a62d1300f248d3181fd17","created_at":"2026-10-04T18:54:07.744Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2a02378227a0a178993867d2b915bba","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:07.752Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-8e9a62d1300f248d3181fd17: start","job_status":"start","queue_duration_s":0.264064,"scheduling_latency_s":0.264081} gitlab # [ 170.942335] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.031Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"8e9a62d1300f248d3181fd17","created_at":"2026-10-04T18:54:07.744Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2a02378227a0a178993867d2b915bba","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:07.752Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-8e9a62d1300f248d3181fd17: done: 0.015095 sec","job_status":"done","queue_duration_s":0.264064,"scheduling_latency_s":0.264081,"redis_calls":12,"redis_duration_s":0.001679,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00028,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000786,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000613,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008366,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000208965,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.015095,"completed_at":"2026-10-04T18:54:08.031Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000459,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 170.992861] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.032Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"47b9e0e9e7b458067844b9ba","created_at":"2026-10-04T18:54:07.775Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a00d1f88ef25f01fffe1e2184dfa3f43","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:07.783Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-47b9e0e9e7b458067844b9ba: start","job_status":"start","queue_duration_s":0.248906,"scheduling_latency_s":0.248919} gitlab # [ 170.993717] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.052Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"47b9e0e9e7b458067844b9ba","created_at":"2026-10-04T18:54:07.775Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a00d1f88ef25f01fffe1e2184dfa3f43","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:07.783Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-47b9e0e9e7b458067844b9ba: done: 0.020297 sec","job_status":"done","queue_duration_s":0.248906,"scheduling_latency_s":0.248919,"redis_calls":10,"redis_duration_s":0.001897,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000519,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001029,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000349,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011176,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000270704,"duration_s":0.020297,"completed_at":"2026-10-04T18:54:08.052Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003357,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.066123] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.054Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"6649a2aa3af0c20703aac52c","created_at":"2026-10-04T18:54:07.802Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"55cb45feb75bc6e79ae4346ae87c79be","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:07.810Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-6649a2aa3af0c20703aac52c: start","job_status":"start","queue_duration_s":0.243907,"scheduling_latency_s":0.24392} gitlab # [ 171.066856] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.116Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"6649a2aa3af0c20703aac52c","created_at":"2026-10-04T18:54:07.802Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"55cb45feb75bc6e79ae4346ae87c79be","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:07.810Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-6649a2aa3af0c20703aac52c: done: 0.062742 sec","job_status":"done","queue_duration_s":0.243907,"scheduling_latency_s":0.24392,"redis_calls":12,"redis_duration_s":0.002545,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000358,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00126,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000927,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036737,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000776077,"exclusive_lock_hold_duration_s":0.031843153,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.062742,"completed_at":"2026-10-04T18:54:08.116Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018688,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.112915] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.118Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"c42017a511acc70795adb406","created_at":"2026-10-04T18:54:07.813Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"897db516a6094c2ce127249d944c0ca3","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:07.818Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-c42017a511acc70795adb406: start","job_status":"start","queue_duration_s":0.299436,"scheduling_latency_s":0.299451} gitlab # [ 171.113578] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.139Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"c42017a511acc70795adb406","created_at":"2026-10-04T18:54:07.813Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"897db516a6094c2ce127249d944c0ca3","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:07.818Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-c42017a511acc70795adb406: done: 0.020754 sec","job_status":"done","queue_duration_s":0.299436,"scheduling_latency_s":0.299451,"redis_calls":10,"redis_duration_s":0.001461,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000286,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000867,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000308,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010874,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000231314,"duration_s":0.020754,"completed_at":"2026-10-04T18:54:08.138Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001481,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.154404] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.140Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"53ba63b8a278e663482273b3","created_at":"2026-10-04T18:54:07.825Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"899e09111fbd55601847f64b91c51378","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-04T18:54:07.837Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-53ba63b8a278e663482273b3: start","job_status":"start","queue_duration_s":0.302336,"scheduling_latency_s":0.30235} gitlab # [ 171.155174] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.147Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 171.155570] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.151Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"53ba63b8a278e663482273b3","created_at":"2026-10-04T18:54:07.825Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"899e09111fbd55601847f64b91c51378","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-04T18:54:07.837Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-53ba63b8a278e663482273b3: done: 0.011263 sec","job_status":"done","queue_duration_s":0.302336,"scheduling_latency_s":0.30235,"redis_calls":10,"redis_duration_s":0.001326,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000277,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000791,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000258,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00667,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000255339,"duration_s":0.011263,"completed_at":"2026-10-04T18:54:08.151Z","load_balancing_strategy":"primary","db_duration_s":0.000289,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.208100] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.152Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"d59a33f5ee915afb9033842c","created_at":"2026-10-04T18:54:07.845Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"52cdd3a457c9aa7126dd19ba80dfc8e4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-04T18:54:07.854Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-d59a33f5ee915afb9033842c: start","job_status":"start","queue_duration_s":0.297546,"scheduling_latency_s":0.297558} gitlab # [ 171.208751] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.162Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 171.209491] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.167Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"d59a33f5ee915afb9033842c","created_at":"2026-10-04T18:54:07.845Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"52cdd3a457c9aa7126dd19ba80dfc8e4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-04T18:54:07.854Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-d59a33f5ee915afb9033842c: done: 0.014965 sec","job_status":"done","queue_duration_s":0.297546,"scheduling_latency_s":0.297558,"redis_calls":10,"redis_duration_s":0.001521,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000277,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000871,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000373,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008584,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000229917,"duration_s":0.014965,"completed_at":"2026-10-04T18:54:08.167Z","load_balancing_strategy":"primary","db_duration_s":0.001561,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.272642] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.168Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"ec4321ad628024d5f2a6dbc2","created_at":"2026-10-04T18:54:07.864Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1cf09766d1b5c0e6f530ccf8ecebee69","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:07.871Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-ec4321ad628024d5f2a6dbc2: start","job_status":"start","queue_duration_s":0.296525,"scheduling_latency_s":0.29654} gitlab # [ 171.273480] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.182Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"ec4321ad628024d5f2a6dbc2","created_at":"2026-10-04T18:54:07.864Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1cf09766d1b5c0e6f530ccf8ecebee69","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:07.871Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-ec4321ad628024d5f2a6dbc2: done: 0.014286 sec","job_status":"done","queue_duration_s":0.296525,"scheduling_latency_s":0.29654,"redis_calls":12,"redis_duration_s":0.001767,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000517,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000865,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000385,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00836,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000243885,"duration_s":0.014286,"completed_at":"2026-10-04T18:54:08.182Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000584,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.303161] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.183Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"1feb00357d86d62c1825419a","created_at":"2026-10-04T18:54:07.877Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f75abf889f5cc5a130e1e23f3740d55","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:07.885Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-1feb00357d86d62c1825419a: start","job_status":"start","queue_duration_s":0.298029,"scheduling_latency_s":0.298042} gitlab # [ 171.339457] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.428Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"1feb00357d86d62c1825419a","created_at":"2026-10-04T18:54:07.877Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f75abf889f5cc5a130e1e23f3740d55","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:07.885Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-1feb00357d86d62c1825419a: done: 0.24497 sec","job_status":"done","queue_duration_s":0.298029,"scheduling_latency_s":0.298042,"redis_calls":565,"redis_duration_s":0.07271999999999999,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000267,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000883,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.07157,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.192969,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000247238,"duration_s":0.24497,"completed_at":"2026-10-04T18:54:08.428Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000249,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.391865] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.429Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"4942628d535add43e60840f0","created_at":"2026-10-04T18:54:07.888Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2fca5a1e0ac980ddb4844bc79bb2d75c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:07.894Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-4942628d535add43e60840f0: start","job_status":"start","queue_duration_s":0.535506,"scheduling_latency_s":0.535523} gitlab # [ 171.392625] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.439Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 171.393151] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.444Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"4942628d535add43e60840f0","created_at":"2026-10-04T18:54:07.888Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2fca5a1e0ac980ddb4844bc79bb2d75c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:07.894Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-4942628d535add43e60840f0: done: 0.014928 sec","job_status":"done","queue_duration_s":0.535506,"scheduling_latency_s":0.535523,"redis_calls":10,"redis_duration_s":0.001411,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000278,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000804,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000329,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007915,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000236902,"duration_s":0.014928,"completed_at":"2026-10-04T18:54:08.444Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001474,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.435075] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.445Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"528044bbab26152bae99b5f7","created_at":"2026-10-04T18:54:07.905Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2db401991bf81de870064e9ba82a5ec6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-04T18:54:07.910Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-528044bbab26152bae99b5f7: start","job_status":"start","queue_duration_s":0.535644,"scheduling_latency_s":0.535658} gitlab # [ 171.435667] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.467Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"528044bbab26152bae99b5f7","created_at":"2026-10-04T18:54:07.905Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2db401991bf81de870064e9ba82a5ec6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-04T18:54:07.910Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-528044bbab26152bae99b5f7: done: 0.021727 sec","job_status":"done","queue_duration_s":0.535644,"scheduling_latency_s":0.535658,"redis_calls":10,"redis_duration_s":0.001401,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000261,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000834,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000306,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013149,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000215391,"duration_s":0.021727,"completed_at":"2026-10-04T18:54:08.467Z","load_balancing_strategy":"primary","db_duration_s":0.002855,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.475843] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.468Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"475fc93da9e77ecaae85123e","created_at":"2026-10-04T18:54:07.919Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"27a37fb586dd395d21958dabb17bb821","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:07.928Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-475fc93da9e77ecaae85123e: start","job_status":"start","queue_duration_s":0.540165,"scheduling_latency_s":0.540179} gitlab # [ 171.476695] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.488Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"475fc93da9e77ecaae85123e","created_at":"2026-10-04T18:54:07.919Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"27a37fb586dd395d21958dabb17bb821","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:07.928Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-475fc93da9e77ecaae85123e: done: 0.020158 sec","job_status":"done","queue_duration_s":0.540165,"scheduling_latency_s":0.540179,"redis_calls":13,"redis_duration_s":0.001872,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000261,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001014,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000597,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012618,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00662207,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.020158,"completed_at":"2026-10-04T18:54:08.488Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002184,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 171.539889] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.490Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"e1cc5d621229dae412de9cc7","created_at":"2026-10-04T18:54:07.935Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e116791de39f8b820502dde2fc0e51d9","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:07.940Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-e1cc5d621229dae412de9cc7: start","job_status":"start","queue_duration_s":0.549509,"scheduling_latency_s":0.549525} gitlab # [ 171.540601] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.503Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"e1cc5d621229dae412de9cc7","created_at":"2026-10-04T18:54:07.935Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e116791de39f8b820502dde2fc0e51d9","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:07.940Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-e1cc5d621229dae412de9cc7: done: 0.013937 sec","job_status":"done","queue_duration_s":0.549509,"scheduling_latency_s":0.549525,"redis_calls":12,"redis_duration_s":0.0016769999999999999,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000275,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000779,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000623,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000342501,"exclusive_lock_hold_duration_s":0.001703849,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.013937,"completed_at":"2026-10-04T18:54:08.503Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000932,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.597786] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.505Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"d47f071d435f248e747a529f","created_at":"2026-10-04T18:54:07.946Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e8e7512998fa079977c5f2f53bbf2d2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:07.956Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-d47f071d435f248e747a529f: start","job_status":"start","queue_duration_s":0.548516,"scheduling_latency_s":0.54853} gitlab # [ 171.598593] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.512Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 171.599266] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.516Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"d47f071d435f248e747a529f","created_at":"2026-10-04T18:54:07.946Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e8e7512998fa079977c5f2f53bbf2d2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:07.956Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-d47f071d435f248e747a529f: done: 0.011805 sec","job_status":"done","queue_duration_s":0.548516,"scheduling_latency_s":0.54853,"redis_calls":10,"redis_duration_s":0.001265,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000264,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000749,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000252,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006749,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00020645,"duration_s":0.011805,"completed_at":"2026-10-04T18:54:08.516Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000239,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.645096] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.517Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"cb5f90d47af4f513b8a88a93","created_at":"2026-10-04T18:54:07.963Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a87c9ff7f1a57efdeb1b67f9df1fd3ef","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:07.968Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-cb5f90d47af4f513b8a88a93: start","job_status":"start","queue_duration_s":0.549219,"scheduling_latency_s":0.549232} gitlab # [ 171.645727] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.533Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"cb5f90d47af4f513b8a88a93","created_at":"2026-10-04T18:54:07.963Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a87c9ff7f1a57efdeb1b67f9df1fd3ef","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:07.968Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-cb5f90d47af4f513b8a88a93: done: 0.015571 sec","job_status":"done","queue_duration_s":0.549219,"scheduling_latency_s":0.549232,"redis_calls":12,"redis_duration_s":0.0017109999999999998,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000318,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000757,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000636,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009049,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000339149,"exclusive_lock_hold_duration_s":0.0033241649999999996,"duration_s":0.015571,"completed_at":"2026-10-04T18:54:08.533Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001541,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.698697] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.534Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"fe6764a1f309fd1f52dd0d64","created_at":"2026-10-04T18:54:07.973Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a4833bf672ad9f2d5e45e80fb3bca022","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:07.979Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-fe6764a1f309fd1f52dd0d64: start","job_status":"start","queue_duration_s":0.555339,"scheduling_latency_s":0.555352} gitlab # [ 171.699514] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.541Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 171.700107] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.546Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"fe6764a1f309fd1f52dd0d64","created_at":"2026-10-04T18:54:07.973Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a4833bf672ad9f2d5e45e80fb3bca022","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:07.979Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-fe6764a1f309fd1f52dd0d64: done: 0.011939 sec","job_status":"done","queue_duration_s":0.555339,"scheduling_latency_s":0.555352,"redis_calls":10,"redis_duration_s":0.001313,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000263,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000783,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000267,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006871,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00020673,"duration_s":0.011939,"completed_at":"2026-10-04T18:54:08.546Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000265,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.730667] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.547Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"55b1f2455cc73d175b81473d","created_at":"2026-10-04T18:54:07.995Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ff92c8d66b5938a629969878e8a2f800","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.001Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-55b1f2455cc73d175b81473d: start","job_status":"start","queue_duration_s":0.545857,"scheduling_latency_s":0.54587} gitlab # [ 171.778102] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.559Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"55b1f2455cc73d175b81473d","created_at":"2026-10-04T18:54:07.995Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ff92c8d66b5938a629969878e8a2f800","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.001Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-55b1f2455cc73d175b81473d: done: 0.01262 sec","job_status":"done","queue_duration_s":0.545857,"scheduling_latency_s":0.54587,"redis_calls":12,"redis_duration_s":0.001592,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000313,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000286,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000739,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000254,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007309,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000203378,"duration_s":0.01262,"completed_at":"2026-10-04T18:54:08.559Z","load_balancing_strategy":"primary","db_duration_s":0.000277,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.782098] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.560Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50f7266763bee1848d119c5d","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50f7266763bee1848d119c5d: start","job_status":"start","queue_duration_s":0.557005,"scheduling_latency_s":0.557018} gitlab # [ 171.827562] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.575Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50f7266763bee1848d119c5d","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50f7266763bee1848d119c5d: done: 0.01532 sec","job_status":"done","queue_duration_s":0.557005,"scheduling_latency_s":0.557018,"redis_calls":16,"redis_duration_s":0.002421,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000344,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000705,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001372,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009191,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000403682,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01532,"completed_at":"2026-10-04T18:54:08.575Z","load_balancing_strategy":"primary","db_duration_s":0.000421,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.868686] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.577Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2e4d6b1874bcf685b7969796","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2e4d6b1874bcf685b7969796: start","job_status":"start","queue_duration_s":0.573415,"scheduling_latency_s":0.573429} gitlab # [ 171.869640] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.587Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2e4d6b1874bcf685b7969796","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2e4d6b1874bcf685b7969796: done: 0.010024 sec","job_status":"done","queue_duration_s":0.573415,"scheduling_latency_s":0.573429,"redis_calls":13,"redis_duration_s":0.0016200000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000559,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001061,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006621,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010024,"completed_at":"2026-10-04T18:54:08.587Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.878684] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.588Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4507062269e2f8fd1b99d792","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4507062269e2f8fd1b99d792: start","job_status":"start","queue_duration_s":0.584343,"scheduling_latency_s":0.584356} gitlab # [ 171.890632] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.597Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4507062269e2f8fd1b99d792","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4507062269e2f8fd1b99d792: done: 0.009945 sec","job_status":"done","queue_duration_s":0.584343,"scheduling_latency_s":0.584356,"redis_calls":13,"redis_duration_s":0.001607,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001095,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006408,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009945,"completed_at":"2026-10-04T18:54:08.597Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.928545] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.598Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e174a46f037223826874ca3","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e174a46f037223826874ca3: start","job_status":"start","queue_duration_s":0.595152,"scheduling_latency_s":0.595165} gitlab # [ 171.979679] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.608Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e174a46f037223826874ca3","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e174a46f037223826874ca3: done: 0.010028 sec","job_status":"done","queue_duration_s":0.595152,"scheduling_latency_s":0.595165,"redis_calls":13,"redis_duration_s":0.00169,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001127,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00646,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010028,"completed_at":"2026-10-04T18:54:08.608Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.980697] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.609Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"94d84b01e20dea67f68a23d0","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-94d84b01e20dea67f68a23d0: start","job_status":"start","queue_duration_s":0.606027,"scheduling_latency_s":0.606065} gitlab # [ 172.022934] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.619Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"94d84b01e20dea67f68a23d0","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-94d84b01e20dea67f68a23d0: done: 0.010261 sec","job_status":"done","queue_duration_s":0.606027,"scheduling_latency_s":0.606065,"redis_calls":13,"redis_duration_s":0.001767,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001219,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006502,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010261,"completed_at":"2026-10-04T18:54:08.619Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.023643] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.620Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a7036313ee726e2c95827c5","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a7036313ee726e2c95827c5: start","job_status":"start","queue_duration_s":0.617167,"scheduling_latency_s":0.61718} gitlab # [ 172.071231] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.631Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a7036313ee726e2c95827c5","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a7036313ee726e2c95827c5: done: 0.010538 sec","job_status":"done","queue_duration_s":0.617167,"scheduling_latency_s":0.61718,"redis_calls":13,"redis_duration_s":0.0016680000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001115,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006528,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010538,"completed_at":"2026-10-04T18:54:08.631Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.072479] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.632Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2628feea5ae91a9570c9b052","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2628feea5ae91a9570c9b052: start","job_status":"start","queue_duration_s":0.628601,"scheduling_latency_s":0.628615} gitlab # [ 172.081687] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.642Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2628feea5ae91a9570c9b052","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2628feea5ae91a9570c9b052: done: 0.010028 sec","job_status":"done","queue_duration_s":0.628601,"scheduling_latency_s":0.628615,"redis_calls":13,"redis_duration_s":0.00177,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001234,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00624,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010028,"completed_at":"2026-10-04T18:54:08.642Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.129718] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.643Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fbc9c64bddc87ed43dd44860","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fbc9c64bddc87ed43dd44860: start","job_status":"start","queue_duration_s":0.639505,"scheduling_latency_s":0.639518} gitlab # [ 172.130592] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.652Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fbc9c64bddc87ed43dd44860","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fbc9c64bddc87ed43dd44860: done: 0.009771 sec","job_status":"done","queue_duration_s":0.639505,"scheduling_latency_s":0.639518,"redis_calls":13,"redis_duration_s":0.001604,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001051,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006227,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009771,"completed_at":"2026-10-04T18:54:08.652Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.163233] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.654Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c19293d58a9f4c1bd32d636","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c19293d58a9f4c1bd32d636: start","job_status":"start","queue_duration_s":0.650312,"scheduling_latency_s":0.650326} gitlab # [ 172.174672] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.664Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c19293d58a9f4c1bd32d636","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c19293d58a9f4c1bd32d636: done: 0.010276 sec","job_status":"done","queue_duration_s":0.650312,"scheduling_latency_s":0.650326,"redis_calls":13,"redis_duration_s":0.001785,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000596,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001189,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006354,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010276,"completed_at":"2026-10-04T18:54:08.664Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.226195] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.665Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86e47fdf51828743e6625d03","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86e47fdf51828743e6625d03: start","job_status":"start","queue_duration_s":0.66162,"scheduling_latency_s":0.661634} gitlab # [ 172.226839] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.675Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86e47fdf51828743e6625d03","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86e47fdf51828743e6625d03: done: 0.009836 sec","job_status":"done","queue_duration_s":0.66162,"scheduling_latency_s":0.661634,"redis_calls":13,"redis_duration_s":0.001643,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000565,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001078,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006429,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009836,"completed_at":"2026-10-04T18:54:08.675Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.255497] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.676Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f35161ab2cb4fb57a71a6eb","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f35161ab2cb4fb57a71a6eb: start","job_status":"start","queue_duration_s":0.672479,"scheduling_latency_s":0.672492} gitlab # [ 172.299174] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.686Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f35161ab2cb4fb57a71a6eb","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f35161ab2cb4fb57a71a6eb: done: 0.009997 sec","job_status":"done","queue_duration_s":0.672479,"scheduling_latency_s":0.672492,"redis_calls":13,"redis_duration_s":0.001669,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001107,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006745,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009997,"completed_at":"2026-10-04T18:54:08.686Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.302469] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.687Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c4bcbe1a099a6b1073cd7632","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c4bcbe1a099a6b1073cd7632: start","job_status":"start","queue_duration_s":0.683557,"scheduling_latency_s":0.683571} gitlab # [ 172.342213] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.697Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c4bcbe1a099a6b1073cd7632","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c4bcbe1a099a6b1073cd7632: done: 0.009981 sec","job_status":"done","queue_duration_s":0.683557,"scheduling_latency_s":0.683571,"redis_calls":13,"redis_duration_s":0.001605,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001078,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006501,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009981,"completed_at":"2026-10-04T18:54:08.697Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.348239] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.698Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0ed54e8f044b5f18bd7072d","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0ed54e8f044b5f18bd7072d: start","job_status":"start","queue_duration_s":0.694571,"scheduling_latency_s":0.694585} gitlab # [ 172.388931] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.708Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0ed54e8f044b5f18bd7072d","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0ed54e8f044b5f18bd7072d: done: 0.009969 sec","job_status":"done","queue_duration_s":0.694571,"scheduling_latency_s":0.694585,"redis_calls":13,"redis_duration_s":0.0017469999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001197,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006536,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009969,"completed_at":"2026-10-04T18:54:08.708Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.394114] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.709Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d32a654d8d6f629cf55ded6a","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d32a654d8d6f629cf55ded6a: start","job_status":"start","queue_duration_s":0.705418,"scheduling_latency_s":0.705432} gitlab # [ 172.442317] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.718Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d32a654d8d6f629cf55ded6a","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d32a654d8d6f629cf55ded6a: done: 0.00987 sec","job_status":"done","queue_duration_s":0.705418,"scheduling_latency_s":0.705432,"redis_calls":13,"redis_duration_s":0.0017410000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001213,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006578,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00987,"completed_at":"2026-10-04T18:54:08.718Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.443502] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.719Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e66c4103646331c23366d67","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e66c4103646331c23366d67: start","job_status":"start","queue_duration_s":0.716167,"scheduling_latency_s":0.71618} gitlab # [ 172.451770] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.729Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e66c4103646331c23366d67","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e66c4103646331c23366d67: done: 0.009939 sec","job_status":"done","queue_duration_s":0.716167,"scheduling_latency_s":0.71618,"redis_calls":13,"redis_duration_s":0.0018650000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000546,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001319,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006459,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009939,"completed_at":"2026-10-04T18:54:08.729Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.499762] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.730Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b88d08976de9eddd610c4715","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b88d08976de9eddd610c4715: start","job_status":"start","queue_duration_s":0.726943,"scheduling_latency_s":0.726956} gitlab # [ 172.500729] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.740Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b88d08976de9eddd610c4715","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b88d08976de9eddd610c4715: done: 0.010096 sec","job_status":"done","queue_duration_s":0.726943,"scheduling_latency_s":0.726956,"redis_calls":13,"redis_duration_s":0.0017000000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000569,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001131,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006272,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010096,"completed_at":"2026-10-04T18:54:08.740Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.546610] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.741Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8cb97188ac347d19d1fd382f","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8cb97188ac347d19d1fd382f: start","job_status":"start","queue_duration_s":0.737918,"scheduling_latency_s":0.737932} gitlab # [ 172.547350] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.751Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8cb97188ac347d19d1fd382f","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8cb97188ac347d19d1fd382f: done: 0.009969 sec","job_status":"done","queue_duration_s":0.737918,"scheduling_latency_s":0.737932,"redis_calls":13,"redis_duration_s":0.001704,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000565,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001139,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006277,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009969,"completed_at":"2026-10-04T18:54:08.751Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.583511] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.752Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d1b763b7dd9691430d367e63","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d1b763b7dd9691430d367e63: start","job_status":"start","queue_duration_s":0.74875,"scheduling_latency_s":0.748763} gitlab # [ 172.594917] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.762Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d1b763b7dd9691430d367e63","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d1b763b7dd9691430d367e63: done: 0.009797 sec","job_status":"done","queue_duration_s":0.74875,"scheduling_latency_s":0.748763,"redis_calls":13,"redis_duration_s":0.001758,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000703,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001055,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009797,"completed_at":"2026-10-04T18:54:08.762Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.639400] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.763Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8fef5d0fde7b041eaf3dd107","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8fef5d0fde7b041eaf3dd107: start","job_status":"start","queue_duration_s":0.759572,"scheduling_latency_s":0.759585} gitlab # [ 172.641927] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.773Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8fef5d0fde7b041eaf3dd107","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8fef5d0fde7b041eaf3dd107: done: 0.010394 sec","job_status":"done","queue_duration_s":0.759572,"scheduling_latency_s":0.759585,"redis_calls":13,"redis_duration_s":0.0016510000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001126,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00634,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010394,"completed_at":"2026-10-04T18:54:08.773Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.679154] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.774Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3fddfd784dea590f3027f20c","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3fddfd784dea590f3027f20c: start","job_status":"start","queue_duration_s":0.771005,"scheduling_latency_s":0.771019} gitlab # [ 172.723132] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.784Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3fddfd784dea590f3027f20c","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3fddfd784dea590f3027f20c: done: 0.009891 sec","job_status":"done","queue_duration_s":0.771005,"scheduling_latency_s":0.771019,"redis_calls":13,"redis_duration_s":0.001696,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001151,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006682,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009891,"completed_at":"2026-10-04T18:54:08.784Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.725748] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.785Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcaaf8d7feed18ff0bbc68fb","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcaaf8d7feed18ff0bbc68fb: start","job_status":"start","queue_duration_s":0.781931,"scheduling_latency_s":0.781944} gitlab # [ 172.737459] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.795Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcaaf8d7feed18ff0bbc68fb","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcaaf8d7feed18ff0bbc68fb: done: 0.009899 sec","job_status":"done","queue_duration_s":0.781931,"scheduling_latency_s":0.781944,"redis_calls":13,"redis_duration_s":0.0017209999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001168,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006413,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009899,"completed_at":"2026-10-04T18:54:08.795Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.778530] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.796Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9cd56488f064bd84907af33d","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9cd56488f064bd84907af33d: start","job_status":"start","queue_duration_s":0.792846,"scheduling_latency_s":0.792859} gitlab # [ 172.784519] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.806Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9cd56488f064bd84907af33d","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9cd56488f064bd84907af33d: done: 0.009802 sec","job_status":"done","queue_duration_s":0.792846,"scheduling_latency_s":0.792859,"redis_calls":13,"redis_duration_s":0.001656,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001139,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006367,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009802,"completed_at":"2026-10-04T18:54:08.806Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.821319] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d98f85d4e39e8090698ba5bf","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d98f85d4e39e8090698ba5bf: start","job_status":"start","queue_duration_s":0.803651,"scheduling_latency_s":0.803664} gitlab # [ 172.832840] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.817Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d98f85d4e39e8090698ba5bf","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d98f85d4e39e8090698ba5bf: done: 0.009846 sec","job_status":"done","queue_duration_s":0.803651,"scheduling_latency_s":0.803664,"redis_calls":13,"redis_duration_s":0.001704,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000578,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001126,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006391,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009846,"completed_at":"2026-10-04T18:54:08.817Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.873989] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.818Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e57e799e0a899616067bc25","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e57e799e0a899616067bc25: start","job_status":"start","queue_duration_s":0.814376,"scheduling_latency_s":0.81439} gitlab # [ 172.879631] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.828Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e57e799e0a899616067bc25","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e57e799e0a899616067bc25: done: 0.010095 sec","job_status":"done","queue_duration_s":0.814376,"scheduling_latency_s":0.81439,"redis_calls":13,"redis_duration_s":0.001682,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000579,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001103,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006522,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010095,"completed_at":"2026-10-04T18:54:08.828Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.921202] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.829Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4135067e9706c1a34b14ab72","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4135067e9706c1a34b14ab72: start","job_status":"start","queue_duration_s":0.825349,"scheduling_latency_s":0.825363} gitlab # [ 172.965466] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.839Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4135067e9706c1a34b14ab72","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4135067e9706c1a34b14ab72: done: 0.01006 sec","job_status":"done","queue_duration_s":0.825349,"scheduling_latency_s":0.825363,"redis_calls":13,"redis_duration_s":0.001716,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000605,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001111,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006289,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01006,"completed_at":"2026-10-04T18:54:08.839Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.966682] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.839Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b374ba874cdbe8ac5a3ffb3e","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b374ba874cdbe8ac5a3ffb3e: start","job_status":"start","queue_duration_s":0.836272,"scheduling_latency_s":0.836285} gitlab # [ 172.974572] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.849Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b374ba874cdbe8ac5a3ffb3e","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b374ba874cdbe8ac5a3ffb3e: done: 0.009978 sec","job_status":"done","queue_duration_s":0.836272,"scheduling_latency_s":0.836285,"redis_calls":13,"redis_duration_s":0.001692,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000559,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001133,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006285,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009978,"completed_at":"2026-10-04T18:54:08.849Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.034779] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.850Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7a07d10f545629850d18562","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7a07d10f545629850d18562: start","job_status":"start","queue_duration_s":0.847109,"scheduling_latency_s":0.847122} gitlab # [ 173.035705] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.860Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7a07d10f545629850d18562","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7a07d10f545629850d18562: done: 0.010149 sec","job_status":"done","queue_duration_s":0.847109,"scheduling_latency_s":0.847122,"redis_calls":13,"redis_duration_s":0.0016950000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000621,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001074,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006287,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010149,"completed_at":"2026-10-04T18:54:08.860Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.084640] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.861Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57164109667c06acaf774153","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57164109667c06acaf774153: start","job_status":"start","queue_duration_s":0.85814,"scheduling_latency_s":0.858154} gitlab # [ 173.085589] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.871Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57164109667c06acaf774153","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57164109667c06acaf774153: done: 0.010105 sec","job_status":"done","queue_duration_s":0.85814,"scheduling_latency_s":0.858154,"redis_calls":13,"redis_duration_s":0.001947,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000609,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001338,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006354,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010105,"completed_at":"2026-10-04T18:54:08.871Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.137744] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.872Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d4020fe088813a374d96b1b","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d4020fe088813a374d96b1b: start","job_status":"start","queue_duration_s":0.869281,"scheduling_latency_s":0.869294} gitlab # [ 173.138738] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.882Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d4020fe088813a374d96b1b","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d4020fe088813a374d96b1b: done: 0.010037 sec","job_status":"done","queue_duration_s":0.869281,"scheduling_latency_s":0.869294,"redis_calls":13,"redis_duration_s":0.0016970000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001134,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006383,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010037,"completed_at":"2026-10-04T18:54:08.882Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.154916] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.884Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27e7d7b49005b92bccc0765f","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27e7d7b49005b92bccc0765f: start","job_status":"start","queue_duration_s":0.880345,"scheduling_latency_s":0.880359} gitlab # [ 173.166740] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.893Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27e7d7b49005b92bccc0765f","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27e7d7b49005b92bccc0765f: done: 0.009789 sec","job_status":"done","queue_duration_s":0.880345,"scheduling_latency_s":0.880359,"redis_calls":13,"redis_duration_s":0.001642,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006402,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009789,"completed_at":"2026-10-04T18:54:08.893Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.218394] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.894Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d6331625300e9a008bcd626a","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d6331625300e9a008bcd626a: start","job_status":"start","queue_duration_s":0.891121,"scheduling_latency_s":0.891134} gitlab # [ 173.219158] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.904Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d6331625300e9a008bcd626a","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d6331625300e9a008bcd626a: done: 0.009872 sec","job_status":"done","queue_duration_s":0.891121,"scheduling_latency_s":0.891134,"redis_calls":13,"redis_duration_s":0.001791,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000568,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001223,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006508,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009872,"completed_at":"2026-10-04T18:54:08.904Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.250357] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.905Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d407adb4668a4e26eebe2bfc","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d407adb4668a4e26eebe2bfc: start","job_status":"start","queue_duration_s":0.902018,"scheduling_latency_s":0.902059} gitlab # [ 173.299584] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.915Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d407adb4668a4e26eebe2bfc","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d407adb4668a4e26eebe2bfc: done: 0.009746 sec","job_status":"done","queue_duration_s":0.902018,"scheduling_latency_s":0.902059,"redis_calls":13,"redis_duration_s":0.0016089999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001084,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006645,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009746,"completed_at":"2026-10-04T18:54:08.915Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.300542] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.916Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43f17d8fdca002c731d9098d","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43f17d8fdca002c731d9098d: start","job_status":"start","queue_duration_s":0.912771,"scheduling_latency_s":0.912785} gitlab # [ 173.308628] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.926Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43f17d8fdca002c731d9098d","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43f17d8fdca002c731d9098d: done: 0.009611 sec","job_status":"done","queue_duration_s":0.912771,"scheduling_latency_s":0.912785,"redis_calls":13,"redis_duration_s":0.0017980000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000715,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001083,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006505,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009611,"completed_at":"2026-10-04T18:54:08.926Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.344896] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.926Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1cd26a79abe8058a70cbd158","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1cd26a79abe8058a70cbd158: start","job_status":"start","queue_duration_s":0.923219,"scheduling_latency_s":0.923233} gitlab # [ 173.392988] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.936Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1cd26a79abe8058a70cbd158","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1cd26a79abe8058a70cbd158: done: 0.009726 sec","job_status":"done","queue_duration_s":0.923219,"scheduling_latency_s":0.923233,"redis_calls":13,"redis_duration_s":0.0016359999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001106,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006573,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009726,"completed_at":"2026-10-04T18:54:08.936Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.393971] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.937Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1706955442fde46961488ba8","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1706955442fde46961488ba8: start","job_status":"start","queue_duration_s":0.933795,"scheduling_latency_s":0.933807} gitlab # [ 173.401984] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.947Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1706955442fde46961488ba8","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1706955442fde46961488ba8: done: 0.009774 sec","job_status":"done","queue_duration_s":0.933795,"scheduling_latency_s":0.933807,"redis_calls":13,"redis_duration_s":0.001742,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001217,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00656,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009774,"completed_at":"2026-10-04T18:54:08.947Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.474796] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.948Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7910b8366df69393ba239003","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7910b8366df69393ba239003: start","job_status":"start","queue_duration_s":0.944411,"scheduling_latency_s":0.944425} gitlab # [ 173.475213] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.957Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7910b8366df69393ba239003","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7910b8366df69393ba239003: done: 0.00977 sec","job_status":"done","queue_duration_s":0.944411,"scheduling_latency_s":0.944425,"redis_calls":13,"redis_duration_s":0.001643,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001133,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006387,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00977,"completed_at":"2026-10-04T18:54:08.957Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.513876] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.958Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7a5ba95a44c647a316afc54","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7a5ba95a44c647a316afc54: start","job_status":"start","queue_duration_s":0.955146,"scheduling_latency_s":0.955159} gitlab # [ 173.514657] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.969Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7a5ba95a44c647a316afc54","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7a5ba95a44c647a316afc54: done: 0.010283 sec","job_status":"done","queue_duration_s":0.955146,"scheduling_latency_s":0.955159,"redis_calls":13,"redis_duration_s":0.0019690000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000593,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001376,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00626,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010283,"completed_at":"2026-10-04T18:54:08.969Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.556531] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.969Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba3951b714d3e5ea4e3f5f79","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba3951b714d3e5ea4e3f5f79: start","job_status":"start","queue_duration_s":0.966283,"scheduling_latency_s":0.966296} gitlab # [ 173.557282] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba3951b714d3e5ea4e3f5f79","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba3951b714d3e5ea4e3f5f79: done: 0.009906 sec","job_status":"done","queue_duration_s":0.966283,"scheduling_latency_s":0.966296,"redis_calls":13,"redis_duration_s":0.001656,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000585,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001071,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006409,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009906,"completed_at":"2026-10-04T18:54:08.979Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.596186] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.980Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9bf1800a85c8658db83c9908","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9bf1800a85c8658db83c9908: start","job_status":"start","queue_duration_s":0.97719,"scheduling_latency_s":0.977203} gitlab # [ 173.596856] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.990Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9bf1800a85c8658db83c9908","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9bf1800a85c8658db83c9908: done: 0.009628 sec","job_status":"done","queue_duration_s":0.97719,"scheduling_latency_s":0.977203,"redis_calls":13,"redis_duration_s":0.001572,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00105,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006176,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009628,"completed_at":"2026-10-04T18:54:08.990Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.636733] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:08.991Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af0fbeb757f520865eb4733d","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af0fbeb757f520865eb4733d: start","job_status":"start","queue_duration_s":0.988204,"scheduling_latency_s":0.988219} gitlab # [ 173.637615] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:09.001Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af0fbeb757f520865eb4733d","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af0fbeb757f520865eb4733d: done: 0.009671 sec","job_status":"done","queue_duration_s":0.988204,"scheduling_latency_s":0.988219,"redis_calls":13,"redis_duration_s":0.001556,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001037,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006374,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009671,"completed_at":"2026-10-04T18:54:09.001Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.693490] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:09.002Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4cd014146d792815b4c18a4d","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4cd014146d792815b4c18a4d: start","job_status":"start","queue_duration_s":0.998873,"scheduling_latency_s":0.998887} gitlab # [ 173.694361] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:09.012Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4cd014146d792815b4c18a4d","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4cd014146d792815b4c18a4d: done: 0.009733 sec","job_status":"done","queue_duration_s":0.998873,"scheduling_latency_s":0.998887,"redis_calls":13,"redis_duration_s":0.001621,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001102,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006369,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009733,"completed_at":"2026-10-04T18:54:09.012Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.712613] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:09.013Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02fda9403dd2a91a4c51d2af","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02fda9403dd2a91a4c51d2af: start","job_status":"start","queue_duration_s":1.00961,"scheduling_latency_s":1.009623} gitlab # [ 173.750776] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:09.023Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02fda9403dd2a91a4c51d2af","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02fda9403dd2a91a4c51d2af: done: 0.009739 sec","job_status":"done","queue_duration_s":1.00961,"scheduling_latency_s":1.009623,"redis_calls":13,"redis_duration_s":0.001607,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000552,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001055,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006367,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009739,"completed_at":"2026-10-04T18:54:09.022Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.795284] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:09.024Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"818625048c9ad5bdcbe30c26","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-818625048c9ad5bdcbe30c26: start","job_status":"start","queue_duration_s":1.020374,"scheduling_latency_s":1.020389} gitlab # [ 173.795936] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:09.033Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"818625048c9ad5bdcbe30c26","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-818625048c9ad5bdcbe30c26: done: 0.009681 sec","job_status":"done","queue_duration_s":1.020374,"scheduling_latency_s":1.020389,"redis_calls":13,"redis_duration_s":0.001657,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000567,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00109,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006352,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009681,"completed_at":"2026-10-04T18:54:09.033Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.839820] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:09.034Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1b56116ba0ad64948770a099","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1b56116ba0ad64948770a099: start","job_status":"start","queue_duration_s":1.030923,"scheduling_latency_s":1.030936} gitlab # [ 173.840638] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:09.044Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1b56116ba0ad64948770a099","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1b56116ba0ad64948770a099: done: 0.00996 sec","job_status":"done","queue_duration_s":1.030923,"scheduling_latency_s":1.030936,"redis_calls":13,"redis_duration_s":0.001825,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000565,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00126,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00638,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00996,"completed_at":"2026-10-04T18:54:09.044Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.849257] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:09.045Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93d0406dc756ab996d9392aa","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93d0406dc756ab996d9392aa: start","job_status":"start","queue_duration_s":1.041747,"scheduling_latency_s":1.04176} gitlab # [ 173.895827] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:09.055Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93d0406dc756ab996d9392aa","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93d0406dc756ab996d9392aa: done: 0.009862 sec","job_status":"done","queue_duration_s":1.041747,"scheduling_latency_s":1.04176,"redis_calls":13,"redis_duration_s":0.001814,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000558,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001256,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006293,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009862,"completed_at":"2026-10-04T18:54:09.055Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.897044] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:09.056Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e7a620ae508a8c3962e2744","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e7a620ae508a8c3962e2744: start","job_status":"start","queue_duration_s":1.052541,"scheduling_latency_s":1.052555} gitlab # [ 173.905903] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:09.066Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e7a620ae508a8c3962e2744","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e7a620ae508a8c3962e2744: done: 0.010222 sec","job_status":"done","queue_duration_s":1.052541,"scheduling_latency_s":1.052555,"redis_calls":13,"redis_duration_s":0.001947,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000689,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001258,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0062,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010222,"completed_at":"2026-10-04T18:54:09.066Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.948321] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:09.067Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89ea33b140f466ba68e23391","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89ea33b140f466ba68e23391: start","job_status":"start","queue_duration_s":1.063632,"scheduling_latency_s":1.063646} gitlab # [ 173.951599] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:09.077Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89ea33b140f466ba68e23391","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89ea33b140f466ba68e23391: done: 0.009895 sec","job_status":"done","queue_duration_s":1.063632,"scheduling_latency_s":1.063646,"redis_calls":13,"redis_duration_s":0.001608,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000555,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001053,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006176,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009895,"completed_at":"2026-10-04T18:54:09.077Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.005173] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:09.078Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b097a7056d747c008da8161","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b097a7056d747c008da8161: start","job_status":"start","queue_duration_s":1.074396,"scheduling_latency_s":1.07441} gitlab # [ 174.005882] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:09.087Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b097a7056d747c008da8161","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b097a7056d747c008da8161: done: 0.009575 sec","job_status":"done","queue_duration_s":1.074396,"scheduling_latency_s":1.07441,"redis_calls":13,"redis_duration_s":0.0016229999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001098,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00612,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009575,"completed_at":"2026-10-04T18:54:09.087Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.069723] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:09.088Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5cd0684bc103f57222a8e1f","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5cd0684bc103f57222a8e1f: start","job_status":"start","queue_duration_s":1.084968,"scheduling_latency_s":1.084981} gitlab # [ 174.070663] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:09.098Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5cd0684bc103f57222a8e1f","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5cd0684bc103f57222a8e1f: done: 0.009827 sec","job_status":"done","queue_duration_s":1.084968,"scheduling_latency_s":1.084981,"redis_calls":13,"redis_duration_s":0.0018119999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00129,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006362,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009827,"completed_at":"2026-10-04T18:54:09.098Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.109874] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:09.099Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"221af2f30cb85ac79183a661","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-221af2f30cb85ac79183a661: start","job_status":"start","queue_duration_s":1.095809,"scheduling_latency_s":1.095822} gitlab # [ 174.110628] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:09.109Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"221af2f30cb85ac79183a661","created_at":"2026-10-04T18:54:07.876Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-5f5d2c904c124de3","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"79d3cbdb86db1259480b9858b515d8fa","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:08.003Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-221af2f30cb85ac79183a661: done: 0.009699 sec","job_status":"done","queue_duration_s":1.095809,"scheduling_latency_s":1.095822,"redis_calls":13,"redis_duration_s":0.001592,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001079,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006574,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009699,"completed_at":"2026-10-04T18:54:09.109Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.309299] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 1888628664 wd_nsec: 1888628386 gitlab # [ 174.168766] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:11.257Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"cc386941002c309e5e368fc9","created_at":"2026-10-04T18:54:08.007Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f865e49908fec981c4743b83c5d2217","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:11.255Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-cc386941002c309e5e368fc9: start","job_status":"start","queue_duration_s":0.0024,"scheduling_latency_s":0.002434} gitlab # [ 174.273133] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:11.361Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"cc386941002c309e5e368fc9","created_at":"2026-10-04T18:54:08.007Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f865e49908fec981c4743b83c5d2217","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:11.255Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-cc386941002c309e5e368fc9: done: 0.103907 sec","job_status":"done","queue_duration_s":0.0024,"scheduling_latency_s":0.002434,"redis_calls":10,"redis_duration_s":0.011296,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001912,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007011,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002373,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.048,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.048,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017058,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.002213969,"duration_s":0.103907,"completed_at":"2026-10-04T18:54:11.361Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.048239,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.331791] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:11.420Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"1eecebbd287a1d2f156ea395","created_at":"2026-10-04T18:54:11.272Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"48ee21e874dacd71243bcca503352710","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:11.281Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-1eecebbd287a1d2f156ea395: start","job_status":"start","queue_duration_s":0.13901,"scheduling_latency_s":0.139055} gitlab # [ 174.429166] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:11.518Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"1eecebbd287a1d2f156ea395","created_at":"2026-10-04T18:54:11.272Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"48ee21e874dacd71243bcca503352710","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:11.281Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-1eecebbd287a1d2f156ea395: done: 0.097438 sec","job_status":"done","queue_duration_s":0.13901,"scheduling_latency_s":0.139055,"redis_calls":10,"redis_duration_s":0.028459,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001624,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.024608,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002227,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022138,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000865753,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.097438,"completed_at":"2026-10-04T18:54:11.518Z","load_balancing_strategy":"primary","db_duration_s":0.015894,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.460959] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:11.523Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"6c4a962b753ba8b6028e876d","created_at":"2026-10-04T18:54:11.332Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d6842d3c004b9744b89816cc47d3aede","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:11.351Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-6c4a962b753ba8b6028e876d: start","job_status":"start","queue_duration_s":0.171681,"scheduling_latency_s":0.17171} gitlab # [ 174.513511] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:11.601Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"6c4a962b753ba8b6028e876d","created_at":"2026-10-04T18:54:11.332Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d6842d3c004b9744b89816cc47d3aede","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:11.351Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-6c4a962b753ba8b6028e876d: done: 0.078382 sec","job_status":"done","queue_duration_s":0.171681,"scheduling_latency_s":0.17171,"redis_calls":19,"redis_duration_s":0.036123,"redis_read_bytes":21,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00063,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000199,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.032522,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.002772,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019964,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00071266,"duration_s":0.078382,"completed_at":"2026-10-04T18:54:11.601Z","load_balancing_strategy":"primary","db_duration_s":0.002531,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.553830] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:11.642Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"d19afcbdfad1292cc7389a5c","created_at":"2026-10-04T18:54:11.417Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"51baa98e4341c708539031df65cdef0d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:11.439Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-d19afcbdfad1292cc7389a5c: start","job_status":"start","queue_duration_s":0.202937,"scheduling_latency_s":0.202982} gitlab # [ 174.751889] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:11.839Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"d19afcbdfad1292cc7389a5c","created_at":"2026-10-04T18:54:11.417Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"51baa98e4341c708539031df65cdef0d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:11.439Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-d19afcbdfad1292cc7389a5c: done: 0.197183 sec","job_status":"done","queue_duration_s":0.202937,"scheduling_latency_s":0.202982,"redis_calls":14,"redis_duration_s":0.003458,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000878,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001377,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001203,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.037,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.037,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.133077,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00073501,"exclusive_lock_hold_duration_s":0.157686801,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.197183,"completed_at":"2026-10-04T18:54:11.839Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.055356,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.753707] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:11.841Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"3f55a12d776b8d475cd21aef","created_at":"2026-10-04T18:54:11.469Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ea00315a5d367b0521d4579a4b50525","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:11.482Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-3f55a12d776b8d475cd21aef: start","job_status":"start","queue_duration_s":0.359479,"scheduling_latency_s":0.359494} gitlab # [ 174.781569] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:11.869Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"3f55a12d776b8d475cd21aef","created_at":"2026-10-04T18:54:11.469Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ea00315a5d367b0521d4579a4b50525","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:11.482Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-3f55a12d776b8d475cd21aef: done: 0.028131 sec","job_status":"done","queue_duration_s":0.359479,"scheduling_latency_s":0.359494,"redis_calls":16,"redis_duration_s":0.0024749999999999998,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001086,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000897,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000492,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012285,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000218743,"duration_s":0.028131,"completed_at":"2026-10-04T18:54:11.869Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00981,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.844105] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:11.870Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"3ef1a16c4fb342650f1ca667","created_at":"2026-10-04T18:54:11.494Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee3f3284455379ee8cfa9d1f2334009b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:11.504Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-3ef1a16c4fb342650f1ca667: start","job_status":"start","queue_duration_s":0.365866,"scheduling_latency_s":0.365879} gitlab # [ 174.854801] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:11.878Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 174.857778] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:11.882Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"3ef1a16c4fb342650f1ca667","created_at":"2026-10-04T18:54:11.494Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee3f3284455379ee8cfa9d1f2334009b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:11.504Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-3ef1a16c4fb342650f1ca667: done: 0.011847 sec","job_status":"done","queue_duration_s":0.365866,"scheduling_latency_s":0.365879,"redis_calls":10,"redis_duration_s":0.001863,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00058,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000364,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006765,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000260368,"duration_s":0.011847,"completed_at":"2026-10-04T18:54:11.882Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000452,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.905084] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:11.884Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7c52bbd5d2fd6ebd06903b8c","created_at":"2026-10-04T18:54:11.541Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-acbfc08c01c2486f","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d6842d3c004b9744b89816cc47d3aede","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:11.549Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7c52bbd5d2fd6ebd06903b8c: start","job_status":"start","queue_duration_s":0.335022,"scheduling_latency_s":0.335036} gitlab # [ 174.908436] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:11.902Z","message":"No existing merge request to be cleaned up."} gitlab # [ 174.909795] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:11.912Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7c52bbd5d2fd6ebd06903b8c","created_at":"2026-10-04T18:54:11.541Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-acbfc08c01c2486f","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d6842d3c004b9744b89816cc47d3aede","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:11.549Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7c52bbd5d2fd6ebd06903b8c: done: 0.027894 sec","job_status":"done","queue_duration_s":0.335022,"scheduling_latency_s":0.335036,"redis_calls":13,"redis_duration_s":0.00268,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000446,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000962,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001272,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016796,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000298362,"duration_s":0.027894,"completed_at":"2026-10-04T18:54:11.911Z","load_balancing_strategy":"primary","db_duration_s":0.003525,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.950845] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:11.913Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"eaf1cac357350bac2914735b","created_at":"2026-10-04T18:54:11.541Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-acbfc08c01c2486f","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d6842d3c004b9744b89816cc47d3aede","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:11.549Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-eaf1cac357350bac2914735b: start","job_status":"start","queue_duration_s":0.363994,"scheduling_latency_s":0.364008} gitlab # [ 174.993680] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:11.920Z","message":"No existing merge request to be cleaned up."} gitlab # [ 174.994122] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:11.925Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"eaf1cac357350bac2914735b","created_at":"2026-10-04T18:54:11.541Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-acbfc08c01c2486f","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d6842d3c004b9744b89816cc47d3aede","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:11.549Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-eaf1cac357350bac2914735b: done: 0.012052 sec","job_status":"done","queue_duration_s":0.363994,"scheduling_latency_s":0.364008,"redis_calls":10,"redis_duration_s":0.001901,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000808,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001093,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007506,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012052,"completed_at":"2026-10-04T18:54:11.925Z","load_balancing_strategy":"primary","db_duration_s":0.001376,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.995129] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:11.926Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8cd64234f38438d1a7e228a8","created_at":"2026-10-04T18:54:11.541Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-acbfc08c01c2486f","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d6842d3c004b9744b89816cc47d3aede","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:11.549Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8cd64234f38438d1a7e228a8: start","job_status":"start","queue_duration_s":0.377075,"scheduling_latency_s":0.37709} gitlab # [ 175.002744] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:11.938Z","message":"No existing merge request to be cleaned up."} gitlab # [ 175.004423] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:11.944Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8cd64234f38438d1a7e228a8","created_at":"2026-10-04T18:54:11.541Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-acbfc08c01c2486f","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d6842d3c004b9744b89816cc47d3aede","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:11.549Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8cd64234f38438d1a7e228a8: done: 0.017848 sec","job_status":"done","queue_duration_s":0.377075,"scheduling_latency_s":0.37709,"redis_calls":10,"redis_duration_s":0.0016460000000000001,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000621,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001025,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007789,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017848,"completed_at":"2026-10-04T18:54:11.943Z","load_balancing_strategy":"primary","db_duration_s":0.010427,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.075656] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:11.944Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"be8bd5de465bdcd5f9effd2f","created_at":"2026-10-04T18:54:11.541Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-acbfc08c01c2486f","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d6842d3c004b9744b89816cc47d3aede","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:11.549Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-be8bd5de465bdcd5f9effd2f: start","job_status":"start","queue_duration_s":0.395818,"scheduling_latency_s":0.395832} gitlab # [ 175.076500] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:11.952Z","message":"No existing merge request to be cleaned up."} gitlab # [ 175.077032] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:11.961Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"be8bd5de465bdcd5f9effd2f","created_at":"2026-10-04T18:54:11.541Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-acbfc08c01c2486f","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d6842d3c004b9744b89816cc47d3aede","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:11.549Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-be8bd5de465bdcd5f9effd2f: done: 0.016408 sec","job_status":"done","queue_duration_s":0.395818,"scheduling_latency_s":0.395832,"redis_calls":10,"redis_duration_s":0.004149,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002071,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002078,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007776,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016408,"completed_at":"2026-10-04T18:54:11.961Z","load_balancing_strategy":"primary","db_duration_s":0.004041,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.086258] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:11.962Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"5790b8012395b893420e1942","created_at":"2026-10-04T18:54:11.519Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8b91d9ce8da86debb0f19ad3e9678cb9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:11.583Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-5790b8012395b893420e1942: start","job_status":"start","queue_duration_s":0.379375,"scheduling_latency_s":0.379389} gitlab # [ 175.099614] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:12.065Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"5790b8012395b893420e1942","created_at":"2026-10-04T18:54:11.519Z","trace_propagation_headers":{"sentry-trace":"bc349caefece429fb5a5a0cc68680b0c-4db042a7e51f4820","baggage":"sentry-trace_id=bc349caefece429fb5a5a0cc68680b0c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8b91d9ce8da86debb0f19ad3e9678cb9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:11.583Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"madc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-5790b8012395b893420e1942: done: 0.102987 sec","job_status":"done","queue_duration_s":0.379375,"scheduling_latency_s":0.379389,"redis_calls":14,"redis_duration_s":0.002009,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000511,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000749,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000749,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.078,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.078,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019296,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000326857,"exclusive_lock_hold_duration_s":0.08777121099999999,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.102987,"completed_at":"2026-10-04T18:54:12.065Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.078993,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 186.169687] postgres[2430]: [2430] ERROR: relation "table_without_models" does not exist gitlab # [ 186.172496] postgres[2430]: [2430] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 189.163074] runuser[2236]: pam_unix(runuser:session): session closed for user gitlab gitlab # [ 189.164566] setup-gitlab-start[2218]: + systemctl restart gitlab-sidekiq.service gitlab # [ 189.192648] systemd[1]: Stopping gitlab-sidekiq.service... gitlab # [ 189.198741] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:26.287Z","message":"Shutting down"} gitlab # [ 189.199301] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:26.288Z","message":"Scheduler exiting..."} gitlab # [ 189.199999] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:26.288Z","message":"Terminating quiet threads for default capsule"} gitlab # [ 189.200695] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:26.288Z","message":"Scheduler exiting..."} gitlab # [ 189.700861] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:26.789Z","message":"Pausing to allow jobs to finish..."} gitlab # [ 190.204162] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:27.292Z","message":"Bye!"} gitlab # [ 190.230189] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-10-04T18:54:27.319Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1640,"worker_id":"sidekiq_0","memwd_rss_bytes":958672896,"retry":0} gitlab # [ 191.017249] systemd[1]: gitlab-sidekiq.service: Deactivated successfully. gitlab # [ 191.019866] systemd[1]: Stopped gitlab-sidekiq.service. gitlab # [ 191.021657] systemd[1]: gitlab-sidekiq.service: Consumed 57.901s CPU time over 1min 54.199s wall clock time, 975.3M memory peak, 4.4M read from disk. gitlab # [ 191.030673] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 191.034777] setup-gitlab-start[2218]: + mkdir -p /var/lib/secrets gitlab # [ 191.037200] setup-gitlab-start[2218]: + echo nixbot-test-token-of-20-chars gitlab # [ 191.039715] setup-gitlab-start[2218]: + chmod 644 /var/lib/secrets/gitlab-token gitlab # [ 191.042146] setup-gitlab-start[2218]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars gitlab # [ 191.228717] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:28 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 163 gitlab # [ 191.692789] gitaly[916]: time="2026-10-04T18:54:28.781Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M4448YZMAY015PYT7YVJ0JY0 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T18:54:38.010" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:54:28.781" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.714628] gitaly[916]: time="2026-10-04T18:54:28.781Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4448YZMAY015PYT7YVJ0JY0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T18:54:38.010" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:54:28.781" grpc.time_ms=0.384 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.777419] gitaly[916]: time="2026-10-04T18:54:28.866Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M4448YZMAY015PYT7YVJ0JY0 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T18:54:38.095" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:54:28.866" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.799578] gitaly[916]: time="2026-10-04T18:54:28.889Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4448YZMAY015PYT7YVJ0JY0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T18:54:38.095" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:54:28.866" grpc.time_ms=0.559 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.866478] gitaly[916]: time="2026-10-04T18:54:28.955Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4448YZMAY015PYT7YVJ0JY0 diskcache=32653121-45f6-45dd-8acf-821901e6dc9f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-04T18:54:38.145" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:54:28.916" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.886434] gitaly[916]: time="2026-10-04T18:54:28.975Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1578 command.oublock=32 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4448YZMAY015PYT7YVJ0JY0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-04T18:54:38.145" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:54:28.916" grpc.time_ms=39.88 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.914607] gitaly[916]: time="2026-10-04T18:54:29.004Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M4448YZMAY015PYT7YVJ0JY0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T18:54:39.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:54:29.003" pid=916 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 191.914832] gitaly[916]: time="2026-10-04T18:54:29.004Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4448YZMAY015PYT7YVJ0JY0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T18:54:39.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:54:29.003" grpc.time_ms=0.902 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.924585] gitaly[916]: time="2026-10-04T18:54:29.014Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4448YZMAY015PYT7YVJ0JY0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T18:54:39.007" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:54:29.007" grpc.time_ms=6.105 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.972069] gitaly[916]: time="2026-10-04T18:54:29.061Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M4448YZMAY015PYT7YVJ0JY0 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T18:54:39.060" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:54:29.061" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.972603] gitaly[916]: time="2026-10-04T18:54:29.062Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4448YZMAY015PYT7YVJ0JY0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T18:54:39.060" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:54:29.061" grpc.time_ms=0.375 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.009673] gitaly[916]: time="2026-10-04T18:54:29.099Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4448YZMAY015PYT7YVJ0JY0 diskcache=4adea497-636e-417b-acd4-b0f014452b43 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-04T18:54:39.064" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:54:29.064" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.010668] gitaly[916]: time="2026-10-04T18:54:29.099Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1572 command.oublock=32 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4448YZMAY015PYT7YVJ0JY0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-04T18:54:39.064" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:54:29.064" grpc.time_ms=34.821 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.033106] gitaly[916]: time="2026-10-04T18:54:29.122Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M4448YZMAY015PYT7YVJ0JY0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T18:54:39.121" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:54:29.121" pid=916 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 192.033576] gitaly[916]: time="2026-10-04T18:54:29.123Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4448YZMAY015PYT7YVJ0JY0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T18:54:39.121" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:54:29.121" grpc.time_ms=0.836 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.042946] gitaly[916]: time="2026-10-04T18:54:29.132Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4448YZMAY015PYT7YVJ0JY0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T18:54:39.126" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:54:29.126" grpc.time_ms=5.527 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.089239] gitaly[916]: time="2026-10-04T18:54:29.178Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4448YZMAY015PYT7YVJ0JY0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T18:54:39.178" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:54:29.178" grpc.time_ms=0.181 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.110523] gitaly[916]: time="2026-10-04T18:54:29.200Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4448YZMAY015PYT7YVJ0JY0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-04T18:54:39.199" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:54:29.199" grpc.time_ms=0.293 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.611888] gitaly[916]: time="2026-10-04T18:54:29.701Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4448YZMAY015PYT7YVJ0JY0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T18:54:39.009" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:54:29.690" grpc.time_ms=11.161 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.879110] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:29 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 1626 gitlab # [ 192.883744] setup-gitlab-start[2477]: {"id":1,"message":null,"error":null} gitlab # [ 194.187301] sidekiq-cluster[2465]: {"severity":"INFO","time":"2026-10-04T18:54:31.258Z","message":"Starting cluster with 1 processes"} gitlab # [ 198.037937] gitaly[916]: time="2026-10-04T18:54:35.126Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M44495FJWPENESASRYNY1S0Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T18:54:45.114" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:54:35.115" grpc.time_ms=11.538 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.134922] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:35 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 228 gitlab # [ 198.141703] setup-gitlab-start[2218]: + mkdir -p /tmp/test-flake gitlab # [ 198.147528] setup-gitlab-start[2218]: + cd /tmp/test-flake gitlab # [ 198.147683] setup-gitlab-start[2218]: + git init -b master gitlab # [ 198.170076] setup-gitlab-start[2550]: Initialized empty Git repository in /tmp/test-flake/.git/ gitlab # [ 198.172963] setup-gitlab-start[2218]: + git config user.name test gitlab # [ 198.180019] setup-gitlab-start[2218]: + git config user.email test@example.com gitlab # [ 198.184101] setup-gitlab-start[2218]: + cat gitlab # [ 198.188980] setup-gitlab-start[2218]: + git add flake.nix gitlab # [ 198.193918] setup-gitlab-start[2218]: + git commit -m 'initial commit' gitlab # [ 198.218547] setup-gitlab-start[2555]: [master (root-commit) 2e64966] initial commit gitlab # [ 198.219414] setup-gitlab-start[2555]: 1 file changed, 10 insertions(+) gitlab # [ 198.220471] setup-gitlab-start[2555]: create mode 100644 flake.nix gitlab # [ 198.224860] setup-gitlab-start[2218]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git gitlab # [ 198.236228] setup-gitlab-start[2218]: + git push -u origin master gitlab # [ 198.440824] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:35 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 168 gitlab # [ 198.543493] gitaly[916]: time="2026-10-04T18:54:35.632Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444960G9ZSP23KK648K06VV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:54:35.620" grpc.time_ms=12.81 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 198.544090] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:35 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 97 gitlab # [ 199.044404] gitaly[916]: time="2026-10-04T18:54:36.133Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T18:54:46.001" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:54:36.121" grpc.time_ms=11.301 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 199.079412] gitaly[916]: time="2026-10-04T18:54:36.168Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-10-04T18:55:06.057" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-10-04T18:54:36.158" grpc.time_ms=10.291 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 199.182331] gitaly[916]: time="2026-10-04T18:54:36.271Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M444964XGQS1HFFSE980J6X3 duration_ms=264ns method=POST pid=916 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 199.244577] gitaly[916]: time="2026-10-04T18:54:36.334Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M444964XGQS1HFFSE980J6X3 duration_ms=61ns method=POST pid=916 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 199.254441] gitaly[916]: time="2026-10-04T18:54:36.334Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:54:36.006" grpc.time_ms=327.596 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 199.314296] gitaly[916]: time="2026-10-04T18:54:36.403Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:54:36.402" grpc.time_ms=1.029 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 199.360451] gitaly[916]: time="2026-10-04T18:54:36.449Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:54:36.449" grpc.time_ms=0.52 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 199.414940] gitaly[916]: time="2026-10-04T18:54:36.504Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:54:36.504" grpc.time_ms=0.327 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 199.510798] gitaly[916]: time="2026-10-04T18:54:36.600Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T18:54:46.009" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:54:36.590" grpc.time_ms=9.65 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 199.539679] gitaly[916]: time="2026-10-04T18:54:36.622Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1570 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-10-04T18:54:46.029" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:54:36.609" grpc.time_ms=12.763 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 199.559513] gitaly[916]: time="2026-10-04T18:54:36.637Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M444964XGQS1HFFSE980J6X3 duration_ms=86ns method=POST pid=916 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 199.563995] gitaly[916]: time="2026-10-04T18:54:36.637Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:54:36.550" grpc.time_ms=87.692 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 199.581616] gitaly[916]: time="2026-10-04T18:54:36.671Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 diskcache=af891ecf-020d-4af2-80d4-316c80e7b026 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:54:35.927" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 199.582263] gitaly[916]: time="2026-10-04T18:54:36.671Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=228 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13609 command.oublock=32 command.real_time_ms=734 command.spawn_token_fork_ms=0 command.system_time_ms=155 command.user_time_ms=73 component=gitaly.StreamServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:54:35.927" grpc.time_ms=743.385 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 199.583376] gitlab-workhorse[826]: time="2026-10-04T18:54:36Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 199.583837] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:36 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 994 gitlab # [ 199.607517] setup-gitlab-start[2559]: To http://gitlab/root/test-flake.git gitlab # [ 199.607679] setup-gitlab-start[2559]: * [new branch] master -> master gitlab # [ 199.611516] setup-gitlab-start[2559]: branch 'master' set up to track 'origin/master'. gitlab # [ 199.615038] systemd[1]: Finished setup-gitlab.service. gitlab # [ 199.617827] systemd[1]: Starting nixbot CI service... gitlab: (finished: waiting for unit setup-gitlab.service, in 200.66 seconds) gitlab: waiting for unit nixbot.service gitlab # [ 200.882895] nixbot[2645]: {"ts": "2026-10-04T18:54:37.971Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} gitlab # [ 200.917598] nixbot[2645]: {"ts": "2026-10-04T18:54:38.007Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} gitlab # [ 200.949661] nixbot[2645]: {"ts": "2026-10-04T18:54:38.038Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} gitlab # [ 200.960098] nixbot[2645]: {"ts": "2026-10-04T18:54:38.049Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} gitlab # [ 200.969585] nixbot[2645]: {"ts": "2026-10-04T18:54:38.059Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} gitlab # [ 200.983833] nixbot[2645]: {"ts": "2026-10-04T18:54:38.073Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} gitlab # [ 200.989498] nixbot[2645]: {"ts": "2026-10-04T18:54:38.078Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} gitlab # [ 200.997370] nixbot[2645]: {"ts": "2026-10-04T18:54:38.086Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} gitlab # [ 201.006678] nixbot[2645]: {"ts": "2026-10-04T18:54:38.096Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} gitlab # [ 201.014893] nixbot[2645]: {"ts": "2026-10-04T18:54:38.104Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} gitlab # [ 201.022525] nixbot[2645]: {"ts": "2026-10-04T18:54:38.111Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} gitlab # [ 201.033664] nixbot[2645]: {"ts": "2026-10-04T18:54:38.123Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} gitlab # [ 201.039094] nixbot[2645]: {"ts": "2026-10-04T18:54:38.128Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} gitlab # [ 201.047266] nixbot[2645]: {"ts": "2026-10-04T18:54:38.136Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} gitlab # [ 201.053126] nixbot[2645]: {"ts": "2026-10-04T18:54:38.142Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} gitlab # [ 201.061348] nixbot[2645]: {"ts": "2026-10-04T18:54:38.150Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} gitlab # [ 201.064819] nixbot[2645]: {"ts": "2026-10-04T18:54:38.154Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} gitlab # [ 201.074236] nixbot[2645]: {"ts": "2026-10-04T18:54:38.162Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} gitlab # [ 201.083262] nixbot[2645]: {"ts": "2026-10-04T18:54:38.172Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} gitlab # [ 201.090111] nixbot[2645]: {"ts": "2026-10-04T18:54:38.179Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"} gitlab # [ 201.096236] nixbot[2645]: {"ts": "2026-10-04T18:54:38.185Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"} gitlab # [ 201.101927] nixbot[2645]: {"ts": "2026-10-04T18:54:38.191Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"} gitlab # [ 201.106290] nixbot[2645]: {"ts": "2026-10-04T18:54:38.195Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"} gitlab # [ 201.110668] nixbot[2645]: {"ts": "2026-10-04T18:54:38.200Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"} gitlab # [ 201.114661] nixbot[2645]: {"ts": "2026-10-04T18:54:38.204Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"} gitlab # [ 201.119182] nixbot[2645]: {"ts": "2026-10-04T18:54:38.208Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"} gitlab # [ 201.122552] nixbot[2645]: {"ts": "2026-10-04T18:54:38.212Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"} gitlab # [ 201.125309] nixbot[2645]: {"ts": "2026-10-04T18:54:38.214Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"} gitlab # [ 201.130072] nixbot[2645]: {"ts": "2026-10-04T18:54:38.219Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"} gitlab # [ 201.141121] nixbot[2645]: {"ts": "2026-10-04T18:54:38.230Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"} gitlab # [ 201.150449] nixbot[2645]: {"ts": "2026-10-04T18:54:38.239Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"} gitlab # [ 201.154857] nixbot[2645]: {"ts": "2026-10-04T18:54:38.244Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"} gitlab # [ 201.158895] nixbot[2645]: {"ts": "2026-10-04T18:54:38.248Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"} gitlab # [ 201.166457] nixbot[2645]: {"ts": "2026-10-04T18:54:38.255Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"} gitlab # [ 201.170649] nixbot[2645]: {"ts": "2026-10-04T18:54:38.260Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"} gitlab # [ 201.181345] nixbot[2645]: {"ts": "2026-10-04T18:54:38.270Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"} gitlab # [ 201.186409] nixbot[2645]: {"ts": "2026-10-04T18:54:38.275Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 36, "migration": "pr_approvals"} gitlab # [ 201.544796] nixbot[2645]: INFO: Started server process [2645] gitlab # [ 201.545742] nixbot[2645]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit) gitlab # [ 201.566169] nixbot[2645]: INFO: ::ffff:127.0.0.1:38488 - "GET /health HTTP/1.1" 200 OK gitlab # [ 201.574635] systemd[1]: Started nixbot CI service. gitlab # [ 201.577247] systemd[1]: Reached target Multi-User System. gitlab # [ 201.578419] systemd[1]: Startup finished in 1.222s (kernel) + 3.674s (initrd) + 3min 16.681s (userspace) = 3min 21.578s. gitlab # [ 201.775897] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:38 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 209 gitlab # [ 201.782207] nixbot[2645]: {"ts": "2026-10-04T18:54:38.871Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 201.816870] nixbot[2645]: {"ts": "2026-10-04T18:54:38.906Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} gitlab # [ 201.884338] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:38 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 49 gitlab # [ 201.891538] nixbot[2645]: {"ts": "2026-10-04T18:54:38.980Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 201.896893] nixbot[2645]: {"ts": "2026-10-04T18:54:38.982Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} gitlab # [ 202.104620] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:39 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 917 "" "python-httpx/0.28.1" 208 gitlab # [ 202.109769] nixbot[2645]: {"ts": "2026-10-04T18:54:39.195Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""} gitlab: (finished: waiting for unit nixbot.service, in 2.09 seconds) gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health gitlab # [ 202.175443] nixbot[2645]: INFO: ::ffff:127.0.0.1:38496 - "GET /health HTTP/1.1" 200 OK gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.04 seconds) subtest: project discovered and webhook registered gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 202.230478] nixbot[2645]: {"ts": "2026-10-04T18:54:39.319Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 202.235271] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:39 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 114 gitlab # [ 202.270111] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "curl/8.21.0" 48 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.10 seconds) (finished: subtest: project discovered and webhook registered, in 0.10 seconds) subtest: push delivers the webhook gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails' gitlab # [ 202.411155] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "python-httpx/0.28.1" 164 gitlab # [ 202.416362] nixbot[2645]: {"ts": "2026-10-04T18:54:39.500Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 202.490659] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:39 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 917 "" "python-httpx/0.28.1" 75 gitlab # [ 202.495792] nixbot[2645]: {"ts": "2026-10-04T18:54:39.581Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""} gitlab # [ 202.551608] gitaly[916]: time="2026-10-04T18:54:39.640Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M44499Z1NCKBXY93D4WW4RVM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-10-04T18:55:09.523" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:54:39.624" grpc.time_ms=15.171 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 202.584767] gitaly[916]: time="2026-10-04T18:54:39.652Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M44499Z1NCKBXY93D4WW4RVM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T18:55:09.550" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:54:39.651" grpc.time_ms=0.745 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 202.604206] gitaly[916]: time="2026-10-04T18:54:39.663Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M44499Z1NCKBXY93D4WW4RVM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-10-04T18:54:49.004" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:54:39.655" grpc.time_ms=7.698 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 202.616688] nixbot[2645]: {"ts": "2026-10-04T18:54:39.685Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""} gitlab # [ 202.619132] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 97 gitlab # [ 202.679356] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 77 gitlab # [ 202.679987] nixbot[2645]: {"ts": "2026-10-04T18:54:39.769Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 202.693163] nixbot[2645]: {"ts": "2026-10-04T18:54:39.782Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} gitlab # [ 202.858742] gitaly[916]: time="2026-10-04T18:54:39.948Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M4449A81YBK1K7R118YA9FQW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:54:39.935" grpc.time_ms=12.505 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 202.861978] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:39 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 76 gitlab # [ 203.005075] gitaly[916]: time="2026-10-04T18:54:40.094Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4449ABEZRY6NZD8XPGAQ5GJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:54:40.081" pid=916 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc gitlab # [ 203.005430] gitaly[916]: time="2026-10-04T18:54:40.094Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4449ABEZRY6NZD8XPGAQ5GJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:54:40.081" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 203.006528] gitaly[916]: time="2026-10-04T18:54:40.095Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4449ABEZRY6NZD8XPGAQ5GJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:54:40.081" grpc.time_ms=14.02 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 203.007749] gitlab-workhorse[826]: time="2026-10-04T18:54:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=113 gitlab # [ 203.009878] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:40 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 113 gitlab # [ 203.128417] gitaly[916]: time="2026-10-04T18:54:40.217Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4449AGWW5S9AH1FM8FFDK59 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:54:40.197" pid=916 remote_ip="2001:db8:1::1" request_sha=9ccf52cac2bac36f2164e7e179aac358ab5bab78 response_bytes=224 span.kind=server system=grpc gitlab # [ 203.128729] gitaly[916]: time="2026-10-04T18:54:40.218Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1205 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M4449AGWW5S9AH1FM8FFDK59 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:54:40.197" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 203.129817] gitaly[916]: time="2026-10-04T18:54:40.218Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1205 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M4449AGWW5S9AH1FM8FFDK59 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:54:40.197" grpc.time_ms=21.259 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 203.130991] gitlab-workhorse[826]: time="2026-10-04T18:54:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=224 gitlab # [ 203.131735] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:40 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 224 "" "git/2.55.0" 62 gitlab # [ 203.284733] gitaly[916]: time="2026-10-04T18:54:40.374Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M4449ANXHQWF0C7T6GN8PE2G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:54:40.360" grpc.time_ms=13.298 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 203.312161] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:40 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 84 gitlab # [ 203.400454] gitaly[916]: time="2026-10-04T18:54:40.489Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4449ARPHD6ETW7KXYNM6R17 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:54:40.476" pid=916 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 203.422800] gitaly[916]: time="2026-10-04T18:54:40.489Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4449ARPHD6ETW7KXYNM6R17 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:54:40.476" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 203.442236] gitlab-workhorse[826]: time="2026-10-04T18:54:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 203.444319] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:40 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 84 gitlab # [ 203.444676] gitaly[916]: time="2026-10-04T18:54:40.490Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4449ARPHD6ETW7KXYNM6R17 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:54:40.476" grpc.time_ms=13.511 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 203.545396] gitaly[916]: time="2026-10-04T18:54:40.634Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4449AY51BRCQ7EGAVJCH117 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:54:40.627" grpc.time_ms=7.132 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 203.571499] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:40 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 79 gitlab # [ 203.635451] gitaly[916]: time="2026-10-04T18:54:40.724Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4449B0SEGXRPCZACFV7P5ZA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:54:40.705" pid=916 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc gitlab # [ 203.635691] gitaly[916]: time="2026-10-04T18:54:40.724Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1169 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M4449B0SEGXRPCZACFV7P5ZA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:54:40.705" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 203.676168] gitaly[916]: time="2026-10-04T18:54:40.725Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1169 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M4449B0SEGXRPCZACFV7P5ZA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:54:40.705" grpc.time_ms=19.166 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 203.690181] gitlab-workhorse[826]: time="2026-10-04T18:54:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=209 gitlab # [ 203.694112] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:40 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 114 gitlab # [ 203.793427] gitaly[916]: time="2026-10-04T18:54:40.882Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4449B5N013PRGH0MEQR0G7W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T18:55:10.081" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:54:40.881" grpc.time_ms=0.923 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 203.866687] gitaly[916]: time="2026-10-04T18:54:40.956Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=646 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4449B5N013PRGH0MEQR0G7W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-10-04T18:55:10.141" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:54:40.942" grpc.time_ms=13.76 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 203.922798] gitaly[916]: time="2026-10-04T18:54:41.012Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M4449B5N013PRGH0MEQR0G7W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T18:54:50.229" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:54:41.000" grpc.time_ms=11.752 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 203.971305] gitaly[916]: time="2026-10-04T18:54:41.060Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4449B5N013PRGH0MEQR0G7W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T18:54:51.048" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:54:41.048" grpc.time_ms=12.251 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 204.043757] gitaly[916]: time="2026-10-04T18:54:41.133Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4449B5N013PRGH0MEQR0G7W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-04T18:54:51.131" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:54:41.132" grpc.time_ms=0.554 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 204.090309] gitaly[916]: time="2026-10-04T18:54:41.179Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=619 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4449B5N013PRGH0MEQR0G7W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-04T18:54:51.165" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:54:41.166" grpc.time_ms=12.871 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 204.148807] gitaly[916]: time="2026-10-04T18:54:41.238Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4449B5N013PRGH0MEQR0G7W grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:54:41.236" grpc.time_ms=1.218 pid=916 span.kind=server system=grpc gitlab # [ 204.194364] gitaly[916]: time="2026-10-04T18:54:41.283Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4449B5N013PRGH0MEQR0G7W grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:54:41.283" grpc.time_ms=0.586 pid=916 span.kind=server system=grpc gitlab # [ 204.215546] gitaly[916]: time="2026-10-04T18:54:41.305Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4449B5N013PRGH0MEQR0G7W diskcache=a516e4a6-2be9-4880-85d3-95ce172c841b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T18:54:51.183" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:54:41.183" pid=916 ref=refs/keep-around/2e64966c0f092bf33ef3bab01bb475f0f986e356 remote_ip="2001:db8:1::1" revision=2e64966c0f092bf33ef3bab01bb475f0f986e356 span.kind=server system=grpc username=root gitlab # [ 204.215921] gitaly[916]: time="2026-10-04T18:54:41.305Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=106 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5333 command.oublock=8 command.real_time_ms=110 command.spawn_token_fork_ms=2 command.system_time_ms=70 command.user_time_ms=36 component=gitaly.UnaryServerInterceptor correlation_id=01M4449B5N013PRGH0MEQR0G7W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T18:54:51.183" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:54:41.183" grpc.time_ms=121.661 pid=916 ref=refs/keep-around/2e64966c0f092bf33ef3bab01bb475f0f986e356 remote_ip="2001:db8:1::1" revision=2e64966c0f092bf33ef3bab01bb475f0f986e356 span.kind=server system=grpc username=root gitlab # [ 204.220201] gitaly[916]: time="2026-10-04T18:54:41.309Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4449B5N013PRGH0MEQR0G7W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T18:55:11.008" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:54:41.309" grpc.time_ms=0.403 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 204.330194] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:41 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/2e64966c0f092bf33ef3bab01bb475f0f986e356 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 597 gitlab # [ 204.337859] nixbot[2645]: {"ts": "2026-10-04T18:54:41.426Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/2e64966c0f092bf33ef3bab01bb475f0f986e356 \"HTTP/1.1 201 Created\""} gitlab # [ 204.359546] nixbot[2645]: {"ts": "2026-10-04T18:54:41.448Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-2e64966c0f09-0"} gitlab # [ 204.563704] nixbot[2645]: {"ts": "2026-10-04T18:54:41.653Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-2e64966c0f09-0", "argv0": "bwrap"} gitlab # [ 204.656683] systemd[1]: Started Nix Daemon. gitlab # [ 204.725621] nix-daemon[2811]: accepted connection from pid 2806, user nixbot gitlab # [ 209.682200] nix-daemon[2811]: accepted connection from pid 2861, user nixbot gitlab # [ 209.699359] nix-daemon[2811]: accepted connection from pid 2806, user nixbot gitlab # [ 214.608374] nix-daemon[2811]: accepted connection from pid 2806, user nixbot gitlab # [ 214.720258] nix-daemon[2811]: accepted connection from pid 2903, user nixbot gitlab # [ 214.762509] gitaly[916]: time="2026-10-04T18:54:51.851Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.spawn_token_fork_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4449NW8ERJKTYFBN4CFWTRB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T18:55:21.747" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:54:51.848" grpc.time_ms=2.846 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.893974] gitaly[916]: time="2026-10-04T18:54:51.983Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4449NW8ERJKTYFBN4CFWTRB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T18:55:01.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:54:51.971" grpc.time_ms=11.208 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.968485] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:52 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/2e64966c0f092bf33ef3bab01bb475f0f986e356 HTTP/1.1" 201 676 "" "python-httpx/0.28.1" 272 gitlab # [ 214.969704] nixbot[2645]: {"ts": "2026-10-04T18:54:52.058Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/2e64966c0f092bf33ef3bab01bb475f0f986e356 \"HTTP/1.1 201 Created\""} gitlab # [ 215.014108] gitaly[916]: time="2026-10-04T18:54:52.103Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4449P50BPTAC3GF3RNWVND4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T18:55:22.001" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:54:52.102" grpc.time_ms=0.861 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 215.091082] gitaly[916]: time="2026-10-04T18:54:52.180Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M4449P50BPTAC3GF3RNWVND4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T18:55:02.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:54:52.169" grpc.time_ms=10.849 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 215.137446] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:52 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/2e64966c0f092bf33ef3bab01bb475f0f986e356 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 162 gitlab # [ 215.143315] nixbot[2645]: {"ts": "2026-10-04T18:54:52.232Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/2e64966c0f092bf33ef3bab01bb475f0f986e356 \"HTTP/1.1 201 Created\""} gitlab # [ 219.656716] nix-daemon[2811]: accepted connection from pid 2979, user nixbot gitlab # [ 219.703442] sidekiq-cluster[2522]: 2026-10-04T18:54:56.792Z pid=2522 tid=xv6 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2522, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>433561600} gitlab # [ 219.731452] gitaly[916]: time="2026-10-04T18:54:56.820Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4449TR2HYKWS56V21CE4TK3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T18:55:26.518" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:54:56.819" grpc.time_ms=0.81 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 219.770896] gitaly[916]: time="2026-10-04T18:54:56.860Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4449TR2HYKWS56V21CE4TK3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T18:55:06.273" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:54:56.853" grpc.time_ms=7.022 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 219.828782] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:56 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/2e64966c0f092bf33ef3bab01bb475f0f986e356 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 146 gitlab # [ 219.830343] nixbot[2645]: {"ts": "2026-10-04T18:54:56.919Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/2e64966c0f092bf33ef3bab01bb475f0f986e356 \"HTTP/1.1 201 Created\""} gitlab # [ 219.881411] sidekiq-cluster[2522]: W, [2026-10-04T18:54:56.970884 #2522] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 219.919837] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:54:57.009Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 220.129367] gitaly[916]: time="2026-10-04T18:54:57.218Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4449V489N75P2A64HYM5RG5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:54:57.207" grpc.time_ms=10.909 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 220.131624] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:57 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 60 gitlab # [ 220.183964] gitaly[916]: time="2026-10-04T18:54:57.273Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4449V67ENP11JJCZFR7TR67 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:54:57.262" pid=916 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 220.204176] gitaly[916]: time="2026-10-04T18:54:57.273Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4449V67ENP11JJCZFR7TR67 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:54:57.262" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 220.204341] gitlab-workhorse[826]: time="2026-10-04T18:54:57Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 220.205274] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:57 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 50 gitlab # [ 220.206200] gitaly[916]: time="2026-10-04T18:54:57.273Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4449V67ENP11JJCZFR7TR67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:54:57.262" grpc.time_ms=11.322 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 220.300697] gitaly[916]: time="2026-10-04T18:54:57.390Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4449VA1AG5EJ33MDZ48C35S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:54:57.382" grpc.time_ms=7.463 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 220.301584] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:57 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 45 gitlab # [ 220.370362] gitaly[916]: time="2026-10-04T18:54:57.459Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4449VC3HM5YTQ37NV3HBH96 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:54:57.448" pid=916 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc gitlab # [ 220.370749] gitaly[916]: time="2026-10-04T18:54:57.459Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4449VC3HM5YTQ37NV3HBH96 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:54:57.448" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 220.371892] gitaly[916]: time="2026-10-04T18:54:57.460Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4449VC3HM5YTQ37NV3HBH96 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:54:57.448" grpc.time_ms=11.266 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 220.373223] gitlab-workhorse[826]: time="2026-10-04T18:54:57Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 220.373907] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:54:57 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 49 gitlab # [ 221.686126] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:54:58.775Z","message":"GitLab reliable fetch activated!"} gitlab # [ 221.686577] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:54:58.775Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 242.020300] postgres[3208]: [3208] ERROR: relation "table_without_models" does not exist gitlab # [ 242.021393] postgres[3208]: [3208] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 244.462685] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:21.551Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 244.467288] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:21.556Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 244.471967] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:21.561Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 244.472823] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:21.561Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 244.481436] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:21.570Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 42.46 seconds) gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master gitlab # [ 244.839328] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:55:21 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 42 gitlab # [ 244.931647] gitaly[916]: time="2026-10-04T18:55:22.020Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M444AKAE82ZKWANWN567TJJB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=246 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:55:22.008" grpc.time_ms=12.289 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 244.944361] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:55:22 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 203 "" "git/2.55.0" 88 gitlab # Enumerating objects: 5, done. gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. gitlab # Delta compression using up to 4 threads gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 273.00 KiB/s, done. gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) gitlab # [ 245.181502] gitaly[916]: time="2026-10-04T18:55:22.270Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=330 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444AKE6RSXQZZECFJRJ54ZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-10-04T18:55:52.158" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-10-04T18:55:22.259" grpc.time_ms=11.695 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.201289] gitaly[916]: time="2026-10-04T18:55:22.290Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M444AKE6RSXQZZECFJRJ54ZZ duration_ms=88ns method=POST pid=916 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 245.229285] gitaly[916]: time="2026-10-04T18:55:22.318Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M444AKE6RSXQZZECFJRJ54ZZ duration_ms=27ns method=POST pid=916 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 245.232272] gitaly[916]: time="2026-10-04T18:55:22.319Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444AKE6RSXQZZECFJRJ54ZZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:55:22.201" grpc.time_ms=117.912 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 245.279313] gitaly[916]: time="2026-10-04T18:55:22.368Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444AKE6RSXQZZECFJRJ54ZZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:55:22.367" grpc.time_ms=0.859 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 245.327980] gitaly[916]: time="2026-10-04T18:55:22.417Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444AKE6RSXQZZECFJRJ54ZZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:55:22.416" grpc.time_ms=0.51 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 245.366562] gitaly[916]: time="2026-10-04T18:55:22.456Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444AKE6RSXQZZECFJRJ54ZZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:55:22.455" grpc.time_ms=0.44 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 245.456480] gitaly[916]: time="2026-10-04T18:55:22.545Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M444AKE6RSXQZZECFJRJ54ZZ duration_ms=58ns method=POST pid=916 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 245.466475] gitaly[916]: time="2026-10-04T18:55:22.546Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444AKE6RSXQZZECFJRJ54ZZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:55:22.486" grpc.time_ms=59.618 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 245.493988] gitaly[916]: time="2026-10-04T18:55:22.583Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M444AKE6RSXQZZECFJRJ54ZZ diskcache=190d38a2-df9a-4d92-b27c-6f6a6bd6b2d8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:55:22.118" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 245.508708] gitaly[916]: time="2026-10-04T18:55:22.583Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=220 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=876544 command.minflt=13907 command.oublock=32 command.real_time_ms=455 command.spawn_token_fork_ms=0 command.system_time_ms=144 command.user_time_ms=76 component=gitaly.StreamServerInterceptor correlation_id=01M444AKE6RSXQZZECFJRJ54ZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:55:22.118" grpc.time_ms=465.44 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 245.521181] gitlab-workhorse[826]: time="2026-10-04T18:55:22Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 245.523111] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:55:22 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 529 gitlab # To http://gitlab/root/test-flake.git gitlab # 2e64966..860c014 master -> master gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.80 seconds) gitlab: must succeed: git -C /tmp/test-flake rev-parse master gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=860c014d5e7635ba9901c9e9bf9260e2b44a4446' gitlab # [ 245.622143] nixbot[2645]: INFO: ::ffff:127.0.0.1:35518 - "GET /api/repos/gitlab/root/test-flake/builds?commit=860c014d5e7635ba9901c9e9bf9260e2b44a4446 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=860c014d5e7635ba9901c9e9bf9260e2b44a4446', in 0.08 seconds) gitlab # [ 245.987871] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:23.075Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"4dc0be9a7d313d6c7f382536","created_at":"2026-10-04T18:54:29.022Z","trace_propagation_headers":{"sentry-trace":"de753dadad5647a7a7cc4f9363b831d4-0e8c564c7e0f4a89","baggage":"sentry-trace_id=de753dadad5647a7a7cc4f9363b831d4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4448YZMAY015PYT7YVJ0JY0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:29.045Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-4dc0be9a7d313d6c7f382536: start","job_status":"start","queue_duration_s":54.029925,"scheduling_latency_s":54.030079} gitlab # [ 246.186582] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:23.275Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"4dc0be9a7d313d6c7f382536","created_at":"2026-10-04T18:54:29.022Z","trace_propagation_headers":{"sentry-trace":"de753dadad5647a7a7cc4f9363b831d4-0e8c564c7e0f4a89","baggage":"sentry-trace_id=de753dadad5647a7a7cc4f9363b831d4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4448YZMAY015PYT7YVJ0JY0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:29.045Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-4dc0be9a7d313d6c7f382536: done: 0.199818 sec","job_status":"done","queue_duration_s":54.029925,"scheduling_latency_s":54.030079,"redis_calls":25,"redis_duration_s":0.007145,"redis_read_bytes":912,"redis_write_bytes":2805,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.004269,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001532,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001344,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":11,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.103836,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.099536115,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.199818,"completed_at":"2026-10-04T18:55:23.275Z","load_balancing_strategy":"primary","db_duration_s":0.054547,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 246.266657] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:23.277Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"de10fc54883fc2d8fa5eba2c","created_at":"2026-10-04T18:54:29.053Z","trace_propagation_headers":{"sentry-trace":"de753dadad5647a7a7cc4f9363b831d4-0e8c564c7e0f4a89","baggage":"sentry-trace_id=de753dadad5647a7a7cc4f9363b831d4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4448YZMAY015PYT7YVJ0JY0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:29.057Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-de10fc54883fc2d8fa5eba2c: start","job_status":"start","queue_duration_s":54.219842,"scheduling_latency_s":54.219874} gitlab # [ 246.286163] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:23.375Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"de10fc54883fc2d8fa5eba2c","created_at":"2026-10-04T18:54:29.053Z","trace_propagation_headers":{"sentry-trace":"de753dadad5647a7a7cc4f9363b831d4-0e8c564c7e0f4a89","baggage":"sentry-trace_id=de753dadad5647a7a7cc4f9363b831d4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4448YZMAY015PYT7YVJ0JY0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:29.057Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-de10fc54883fc2d8fa5eba2c: done: 0.098123 sec","job_status":"done","queue_duration_s":54.219842,"scheduling_latency_s":54.219874,"redis_calls":14,"redis_duration_s":0.003525,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000792,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001567,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001166,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.064538,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.07465920899999999,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.098123,"completed_at":"2026-10-04T18:55:23.375Z","load_balancing_strategy":"primary","db_duration_s":0.025906,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 246.327502] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:23.376Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"7c1a6f01e9f784bee571ee2f","created_at":"2026-10-04T18:54:29.612Z","trace_propagation_headers":{"sentry-trace":"de753dadad5647a7a7cc4f9363b831d4-0e8c564c7e0f4a89","baggage":"sentry-trace_id=de753dadad5647a7a7cc4f9363b831d4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4448YZMAY015PYT7YVJ0JY0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:29.617Z","job_size_bytes":5,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-7c1a6f01e9f784bee571ee2f: start","job_status":"start","queue_duration_s":53.759509,"scheduling_latency_s":53.759524} gitlab # [ 246.410343] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:23.499Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"7c1a6f01e9f784bee571ee2f","created_at":"2026-10-04T18:54:29.612Z","trace_propagation_headers":{"sentry-trace":"de753dadad5647a7a7cc4f9363b831d4-0e8c564c7e0f4a89","baggage":"sentry-trace_id=de753dadad5647a7a7cc4f9363b831d4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4448YZMAY015PYT7YVJ0JY0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:29.617Z","job_size_bytes":5,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-7c1a6f01e9f784bee571ee2f: done: 0.122675 sec","job_status":"done","queue_duration_s":53.759509,"scheduling_latency_s":53.759524,"redis_calls":12,"redis_duration_s":0.0025789999999999997,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000641,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001475,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000463,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.087538,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000289422,"duration_s":0.122675,"completed_at":"2026-10-04T18:55:23.499Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.02923,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 246.413299] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:23.500Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"50e1f7332c0c7cf7c68b3d50","created_at":"2026-10-04T18:54:29.639Z","trace_propagation_headers":{"sentry-trace":"de753dadad5647a7a7cc4f9363b831d4-0e8c564c7e0f4a89","baggage":"sentry-trace_id=de753dadad5647a7a7cc4f9363b831d4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4448YZMAY015PYT7YVJ0JY0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:29.642Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-50e1f7332c0c7cf7c68b3d50: start","job_status":"start","queue_duration_s":53.858581,"scheduling_latency_s":53.858597} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=860c014d5e7635ba9901c9e9bf9260e2b44a4446' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=860c014d5e7635ba9901c9e9bf9260e2b44a4446', in 0.04 seconds) gitlab # [ 246.597362] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:23.686Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"50e1f7332c0c7cf7c68b3d50","created_at":"2026-10-04T18:54:29.639Z","trace_propagation_headers":{"sentry-trace":"de753dadad5647a7a7cc4f9363b831d4-0e8c564c7e0f4a89","baggage":"sentry-trace_id=de753dadad5647a7a7cc4f9363b831d4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4448YZMAY015PYT7YVJ0JY0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:29.642Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-50e1f7332c0c7cf7c68b3d50: done: 0.185186 sec","job_status":"done","queue_duration_s":53.858581,"scheduling_latency_s":53.858597,"redis_calls":11,"redis_duration_s":0.0028009999999999997,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000632,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001622,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000547,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.155,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.155,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.109492,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00051487,"duration_s":0.185186,"completed_at":"2026-10-04T18:55:23.685Z","load_balancing_strategy":"primary","db_duration_s":0.073084,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 246.672436] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:23.688Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"c0586fd3103e272ee87f6c4d","created_at":"2026-10-04T18:54:29.644Z","trace_propagation_headers":{"sentry-trace":"de753dadad5647a7a7cc4f9363b831d4-0e8c564c7e0f4a89","baggage":"sentry-trace_id=de753dadad5647a7a7cc4f9363b831d4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4448YZMAY015PYT7YVJ0JY0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:29.647Z","job_size_bytes":89,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-c0586fd3103e272ee87f6c4d: start","job_status":"start","queue_duration_s":54.040698,"scheduling_latency_s":54.040725} gitlab # [ 246.688683] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:23.719Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"c0586fd3103e272ee87f6c4d","created_at":"2026-10-04T18:54:29.644Z","trace_propagation_headers":{"sentry-trace":"de753dadad5647a7a7cc4f9363b831d4-0e8c564c7e0f4a89","baggage":"sentry-trace_id=de753dadad5647a7a7cc4f9363b831d4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4448YZMAY015PYT7YVJ0JY0","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:29.647Z","job_size_bytes":89,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-c0586fd3103e272ee87f6c4d: done: 0.031149 sec","job_status":"done","queue_duration_s":54.040698,"scheduling_latency_s":54.040725,"redis_calls":11,"redis_duration_s":0.002741,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000753,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001444,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000544,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017556,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000300318,"duration_s":0.031149,"completed_at":"2026-10-04T18:55:23.719Z","load_balancing_strategy":"primary","db_duration_s":0.005931,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 246.730656] gitaly[916]: time="2026-10-04T18:55:23.761Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T18:55:33.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:55:23.749" grpc.time_ms=11.63 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.731445] nixbot[2645]: INFO: ::ffff:127.0.0.1:35530 - "GET /api/repos/gitlab/root/test-flake/builds?commit=860c014d5e7635ba9901c9e9bf9260e2b44a4446 HTTP/1.1" 200 OK gitlab # [ 246.732252] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:23.720Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 2e64966c0f092bf33ef3bab01bb475f0f986e356 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"19964c6999b7b99d517bdb93","created_at":"2026-10-04T18:54:36.578Z","trace_propagation_headers":{"sentry-trace":"aa276fd2128d4c5b9500f0feb5c745d2-8eb30a4580194d07","baggage":"sentry-trace_id=aa276fd2128d4c5b9500f0feb5c745d2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444964XGQS1HFFSE980J6X3","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:36.581Z","job_size_bytes":185,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-19964c6999b7b99d517bdb93: start","job_status":"start","queue_duration_s":47.138675,"scheduling_latency_s":47.13869} gitlab # [ 246.732827] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:23.731Z","message":"changes: \"0000000000000000000000000000000000000000 2e64966c0f092bf33ef3bab01bb475f0f986e356 refs/heads/master\\n\""} gitlab # [ 246.733514] gitaly[916]: time="2026-10-04T18:55:23.773Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T18:55:33.025" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:55:23.765" grpc.time_ms=7.454 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.734269] gitaly[916]: time="2026-10-04T18:55:23.813Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-10-04T18:55:33.063" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:55:23.803" grpc.time_ms=8.807 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.820700] gitaly[916]: time="2026-10-04T18:55:23.910Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3933 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:55:23.909" grpc.time_ms=0.59 pid=916 span.kind=server system=grpc gitlab # [ 246.854201] gitaly[916]: time="2026-10-04T18:55:23.943Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3935 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:55:23.942" grpc.time_ms=0.606 pid=916 span.kind=server system=grpc gitlab # [ 246.875497] gitaly[916]: time="2026-10-04T18:55:23.965Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 diskcache=fbc2135e-db8a-4437-b2ab-e8079776e855 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T18:55:33.078" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:55:23.819" pid=916 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 246.875729] gitaly[916]: time="2026-10-04T18:55:23.965Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=106 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4812 command.oublock=8 command.real_time_ms=137 command.spawn_token_fork_ms=0 command.system_time_ms=77 command.user_time_ms=29 component=gitaly.UnaryServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T18:55:33.078" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:55:23.819" grpc.time_ms=146.106 pid=916 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 246.890532] gitaly[916]: time="2026-10-04T18:55:23.980Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T18:55:33.239" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:55:23.979" grpc.time_ms=0.311 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.907815] gitaly[916]: time="2026-10-04T18:55:23.997Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=936 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-10-04T18:55:33.246" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:55:23.986" grpc.time_ms=10.739 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.987894] gitaly[916]: time="2026-10-04T18:55:24.077Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=342 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-10-04T18:55:54.071" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:24.071" grpc.time_ms=5.356 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.004933] gitaly[916]: time="2026-10-04T18:55:24.094Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-04T18:55:34.003" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:55:24.094" grpc.time_ms=0.207 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.012265] gitaly[916]: time="2026-10-04T18:55:24.101Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=340 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-10-04T18:55:34.007" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-10-04T18:55:24.097" grpc.time_ms=4.292 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.029177] gitaly[916]: time="2026-10-04T18:55:24.118Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T18:55:34.019" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:55:24.110" grpc.time_ms=8.481 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.040099] gitaly[916]: time="2026-10-04T18:55:24.129Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T18:55:34.031" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:55:24.121" grpc.time_ms=7.797 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.055469] gitaly[916]: time="2026-10-04T18:55:24.145Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=971 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-04T18:55:54.136" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:24.136" grpc.time_ms=8.47 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.435725] gitaly[916]: time="2026-10-04T18:55:24.525Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-10-04T18:55:34.003" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:24.523" grpc.time_ms=1.214 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.519422] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:24.607Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 2e64966c0f092bf33ef3bab01bb475f0f986e356 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"19964c6999b7b99d517bdb93","created_at":"2026-10-04T18:54:36.578Z","trace_propagation_headers":{"sentry-trace":"aa276fd2128d4c5b9500f0feb5c745d2-8eb30a4580194d07","baggage":"sentry-trace_id=aa276fd2128d4c5b9500f0feb5c745d2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444964XGQS1HFFSE980J6X3","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:54:36.581Z","job_size_bytes":185,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-19964c6999b7b99d517bdb93: done: 0.886696 sec","job_status":"done","queue_duration_s":47.138675,"scheduling_latency_s":47.13869,"gitaly_calls":13,"gitaly_duration_s":0.282398,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.038236,"redis_read_bytes":2103,"redis_write_bytes":21750,"redis_cache_calls":15,"redis_cache_duration_s":0.003124,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.005672,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001302,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7838,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003273,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.013245,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.01162,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.043,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.163,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.185,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.043,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.503627,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000374628,"exclusive_lock_hold_duration_s":0.02533255,"duration_s":0.886696,"completed_at":"2026-10-04T18:55:24.607Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.077868,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 247.602187] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:24.691Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"964665184ac7b63108e11223","created_at":"2026-10-04T18:54:41.313Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e6cd0b224dfe4df8","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:41.318Z","job_size_bytes":65,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-964665184ac7b63108e11223: start","job_status":"start","queue_duration_s":43.372775,"scheduling_latency_s":43.372827} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=860c014d5e7635ba9901c9e9bf9260e2b44a4446' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=860c014d5e7635ba9901c9e9bf9260e2b44a4446', in 0.04 seconds) gitlab # [ 247.698352] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:24.787Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"964665184ac7b63108e11223","created_at":"2026-10-04T18:54:41.313Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e6cd0b224dfe4df8","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:41.318Z","job_size_bytes":65,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-964665184ac7b63108e11223: done: 0.096727 sec","job_status":"done","queue_duration_s":43.372775,"scheduling_latency_s":43.372827,"redis_calls":16,"redis_duration_s":0.004866,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002568,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001681,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000617,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.059428,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000292216,"duration_s":0.096727,"completed_at":"2026-10-04T18:55:24.787Z","load_balancing_strategy":"primary","db_duration_s":0.024119,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 247.710528] nixbot[2645]: INFO: ::ffff:127.0.0.1:35534 - "GET /api/repos/gitlab/root/test-flake/builds?commit=860c014d5e7635ba9901c9e9bf9260e2b44a4446 HTTP/1.1" 200 OK gitlab # [ 247.751105] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:24.840Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"ae492a3fb47ae48070928f24","created_at":"2026-10-04T18:54:41.318Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e6cd0b224dfe4df8","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:41.321Z","job_size_bytes":65,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-ae492a3fb47ae48070928f24: start","job_status":"start","queue_duration_s":43.518685,"scheduling_latency_s":43.518712} gitlab # [ 247.833996] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:24.923Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"ae492a3fb47ae48070928f24","created_at":"2026-10-04T18:54:41.318Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e6cd0b224dfe4df8","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:41.321Z","job_size_bytes":65,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-ae492a3fb47ae48070928f24: done: 0.082994 sec","job_status":"done","queue_duration_s":43.518685,"scheduling_latency_s":43.518712,"redis_calls":21,"redis_duration_s":0.00477,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.002169,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001319,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001282,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042652,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000295289,"duration_s":0.082994,"completed_at":"2026-10-04T18:55:24.923Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.031213,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 247.836240] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:24.924Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"6c0a5b7773779646f3a405e4","created_at":"2026-10-04T18:54:41.401Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e6cd0b224dfe4df8","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:41.406Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-6c0a5b7773779646f3a405e4: start","job_status":"start","queue_duration_s":43.5183,"scheduling_latency_s":43.518319} gitlab # [ 247.981825] gitaly[916]: time="2026-10-04T18:55:25.070Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4449B5N013PRGH0MEQR0G7W grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:55:25.070" grpc.time_ms=0.606 pid=916 span.kind=server system=grpc gitlab # [ 248.021739] gitaly[916]: time="2026-10-04T18:55:25.111Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4449B5N013PRGH0MEQR0G7W grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:55:25.110" grpc.time_ms=0.408 pid=916 span.kind=server system=grpc gitlab # [ 248.067225] gitaly[916]: time="2026-10-04T18:55:25.156Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4449B5N013PRGH0MEQR0G7W grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:55:25.156" grpc.time_ms=0.512 pid=916 span.kind=server system=grpc gitlab # [ 248.089450] gitaly[916]: time="2026-10-04T18:55:25.178Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4449B5N013PRGH0MEQR0G7W diskcache=caa4473b-8a55-4336-b1e7-d037d31918fd grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-04T18:55:55.033" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:55:25.033" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.089983] gitaly[916]: time="2026-10-04T18:55:25.179Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=132 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7548 command.oublock=0 command.real_time_ms=132 command.spawn_token_fork_ms=0 command.system_time_ms=91 command.user_time_ms=41 component=gitaly.UnaryServerInterceptor correlation_id=01M4449B5N013PRGH0MEQR0G7W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-04T18:55:55.033" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:55:25.033" grpc.time_ms=145.211 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.209802] gitaly[916]: time="2026-10-04T18:55:25.299Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4449B5N013PRGH0MEQR0G7W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-04T18:55:55.297" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:25.297" grpc.time_ms=1.187 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.279907] gitaly[916]: time="2026-10-04T18:55:25.369Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4449B5N013PRGH0MEQR0G7W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T18:55:55.367" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:25.367" grpc.time_ms=1.068 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.321779] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:25.410Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"6c0a5b7773779646f3a405e4","created_at":"2026-10-04T18:54:41.401Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e6cd0b224dfe4df8","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:41.406Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2522,"job_status":"rescheduled","message":"PipelineProcessWorker JID-6c0a5b7773779646f3a405e4: rescheduled"} gitlab # [ 248.334105] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:25.422Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"6c0a5b7773779646f3a405e4","created_at":"2026-10-04T18:54:41.401Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e6cd0b224dfe4df8","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T18:54:41.406Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-6c0a5b7773779646f3a405e4: done: 0.498303 sec","job_status":"done","queue_duration_s":43.5183,"scheduling_latency_s":43.518319,"gitaly_calls":3,"gitaly_duration_s":0.210731,"redis_calls":120,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.030591,"redis_read_bytes":460,"redis_write_bytes":29297,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001212,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001478,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.010927,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":27,"redis_queues_duration_s":0.004354,"redis_queues_read_bytes":37,"redis_queues_write_bytes":15584,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.007421,"redis_queues_metadata_read_bytes":34,"redis_queues_metadata_write_bytes":3577,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000522,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000764,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002701,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.061,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.019,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.023,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.032,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.1916,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.440756018,"duration_s":0.498303,"completed_at":"2026-10-04T18:55:25.422Z","load_balancing_strategy":"primary","db_duration_s":0.074997,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 248.389915] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:25.424Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","2e64966c0f092bf33ef3bab01bb475f0f986e356 860c014d5e7635ba9901c9e9bf9260e2b44a4446 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"f5fb703bb2c91b9be375a00b","created_at":"2026-10-04T18:55:22.512Z","trace_propagation_headers":{"sentry-trace":"acf4e703890e4ea2ab1bed775db2704e-139b910c493c479e","baggage":"sentry-trace_id=acf4e703890e4ea2ab1bed775db2704e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444AKE6RSXQZZECFJRJ54ZZ","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:22.515Z","job_size_bytes":185,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-f5fb703bb2c91b9be375a00b: start","job_status":"start","queue_duration_s":2.909403,"scheduling_latency_s":2.909429} gitlab # [ 248.390888] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:25.437Z","message":"changes: \"2e64966c0f092bf33ef3bab01bb475f0f986e356 860c014d5e7635ba9901c9e9bf9260e2b44a4446 refs/heads/master\\n\""} gitlab # [ 248.435241] gitaly[916]: time="2026-10-04T18:55:25.524Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444AKE6RSXQZZECFJRJ54ZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T18:55:35.061" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:55:25.512" grpc.time_ms=12.203 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.458079] gitaly[916]: time="2026-10-04T18:55:25.547Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444AKE6RSXQZZECFJRJ54ZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T18:55:35.017" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:55:25.537" grpc.time_ms=9.884 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.463116] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:25.552Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"d5d051b1444e2c4dd360937d","created_at":"2026-10-04T18:55:25.550Z","trace_propagation_headers":{"sentry-trace":"acf4e703890e4ea2ab1bed775db2704e-530de1dcf45a45b7","baggage":"sentry-trace_id=acf4e703890e4ea2ab1bed775db2704e,sentry-release=Unknown"},"correlation_id":"01M444AKE6RSXQZZECFJRJ54ZZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"72367d35d69d60b56cad158f","job_size_bytes":46,"pid":2522,"job_status":"deduplicated","message":"ProjectCacheWorker JID-d5d051b1444e2c4dd360937d: deduplicated: until executing","deduplication.type":"until executing"} gitlab # [ 248.488520] gitaly[916]: time="2026-10-04T18:55:25.577Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M444AKE6RSXQZZECFJRJ54ZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-10-04T18:55:35.045" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:55:25.565" grpc.time_ms=11.981 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.519232] gitaly[916]: time="2026-10-04T18:55:25.608Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=652 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444AKE6RSXQZZECFJRJ54ZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-10-04T18:55:55.595" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:25.595" grpc.time_ms=12.837 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.537147] gitaly[916]: time="2026-10-04T18:55:25.626Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=346 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M444AKE6RSXQZZECFJRJ54ZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-10-04T18:55:35.099" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-10-04T18:55:25.619" grpc.time_ms=6.545 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.569338] gitaly[916]: time="2026-10-04T18:55:25.658Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M444AKE6RSXQZZECFJRJ54ZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T18:55:35.127" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:55:25.647" grpc.time_ms=11.058 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.645478] gitaly[916]: time="2026-10-04T18:55:25.734Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444AKE6RSXQZZECFJRJ54ZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-10-04T18:55:35.212" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:25.733" grpc.time_ms=1.402 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=860c014d5e7635ba9901c9e9bf9260e2b44a4446' gitlab # [ 248.689479] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:25.777Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","2e64966c0f092bf33ef3bab01bb475f0f986e356 860c014d5e7635ba9901c9e9bf9260e2b44a4446 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"f5fb703bb2c91b9be375a00b","created_at":"2026-10-04T18:55:22.512Z","trace_propagation_headers":{"sentry-trace":"acf4e703890e4ea2ab1bed775db2704e-139b910c493c479e","baggage":"sentry-trace_id=acf4e703890e4ea2ab1bed775db2704e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444AKE6RSXQZZECFJRJ54ZZ","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:22.515Z","job_size_bytes":185,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-f5fb703bb2c91b9be375a00b: done: 0.353093 sec","job_status":"done","queue_duration_s":2.909403,"scheduling_latency_s":2.909429,"gitaly_calls":7,"gitaly_duration_s":0.171991,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.024141,"redis_read_bytes":501,"redis_write_bytes":11944,"redis_cache_calls":9,"redis_cache_duration_s":0.002424,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.001043,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5721,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.0042,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.008447,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.008027,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.025,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.025,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.151651,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000538616,"exclusive_lock_hold_duration_s":0.079013114,"duration_s":0.353093,"completed_at":"2026-10-04T18:55:25.777Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.02823,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 248.747208] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:25.779Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"72367d35d69d60b56cad158f","created_at":"2026-10-04T18:55:23.775Z","trace_propagation_headers":{"sentry-trace":"aa276fd2128d4c5b9500f0feb5c745d2-6a649ce8e2c349cd","baggage":"sentry-trace_id=aa276fd2128d4c5b9500f0feb5c745d2,sentry-release=Unknown"},"correlation_id":"01M444964XGQS1HFFSE980J6X3","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:23.789Z","job_size_bytes":46,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-72367d35d69d60b56cad158f: start","job_status":"start","queue_duration_s":1.99025,"scheduling_latency_s":1.990268} gitlab # [ 248.748289] gitaly[916]: time="2026-10-04T18:55:25.816Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-10-05T00:55:25.810" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:55:25.811" grpc.time_ms=4.794 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.749198] gitaly[916]: time="2026-10-04T18:55:25.834Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=649 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-10-04T18:55:55.822" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:25.823" grpc.time_ms=11.229 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.770074] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:25.859Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"72367d35d69d60b56cad158f","created_at":"2026-10-04T18:55:23.775Z","trace_propagation_headers":{"sentry-trace":"aa276fd2128d4c5b9500f0feb5c745d2-6a649ce8e2c349cd","baggage":"sentry-trace_id=aa276fd2128d4c5b9500f0feb5c745d2,sentry-release=Unknown"},"correlation_id":"01M444964XGQS1HFFSE980J6X3","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:23.789Z","job_size_bytes":46,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-72367d35d69d60b56cad158f: done: 0.079908 sec","job_status":"done","queue_duration_s":1.99025,"scheduling_latency_s":1.990268,"gitaly_calls":2,"gitaly_duration_s":0.019903,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.008635,"redis_read_bytes":69,"redis_write_bytes":4888,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001695,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000543,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001486,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.004198,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000713,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03503,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000278247,"duration_s":0.079908,"completed_at":"2026-10-04T18:55:25.859Z","load_balancing_strategy":"primary","db_duration_s":0.016232,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 248.818633] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:25.860Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"e6511c706a85b57c59ff8161","created_at":"2026-10-04T18:55:23.798Z","trace_propagation_headers":{"sentry-trace":"aa276fd2128d4c5b9500f0feb5c745d2-6a649ce8e2c349cd","baggage":"sentry-trace_id=aa276fd2128d4c5b9500f0feb5c745d2,sentry-release=Unknown"},"correlation_id":"01M444964XGQS1HFFSE980J6X3","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:23.801Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-e6511c706a85b57c59ff8161: start","job_status":"start","queue_duration_s":2.059764,"scheduling_latency_s":2.059779} gitlab # [ 248.823713] gitaly[916]: time="2026-10-04T18:55:25.913Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 diskcache=c6a6367b-1148-4a42-b909-e2d75cb15a24 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-05T00:55:25.877" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:25.878" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.824632] gitaly[916]: time="2026-10-04T18:55:25.913Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1280 command.oublock=0 command.real_time_ms=35 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-05T00:55:25.877" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:25.878" grpc.time_ms=34.81 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=860c014d5e7635ba9901c9e9bf9260e2b44a4446', in 0.17 seconds) gitlab # [ 248.869839] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:25.958Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"e6511c706a85b57c59ff8161","created_at":"2026-10-04T18:55:23.798Z","trace_propagation_headers":{"sentry-trace":"aa276fd2128d4c5b9500f0feb5c745d2-6a649ce8e2c349cd","baggage":"sentry-trace_id=aa276fd2128d4c5b9500f0feb5c745d2,sentry-release=Unknown"},"correlation_id":"01M444964XGQS1HFFSE980J6X3","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:23.801Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-e6511c706a85b57c59ff8161: done: 0.098045 sec","job_status":"done","queue_duration_s":2.059764,"scheduling_latency_s":2.059779,"gitaly_calls":1,"gitaly_duration_s":0.03738,"redis_calls":15,"redis_duration_s":0.004279,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000458,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001719,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00066,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001442,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.011,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.022,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040065,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.076187338,"duration_s":0.098045,"completed_at":"2026-10-04T18:55:25.958Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014178,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.875560] nixbot[2645]: INFO: ::ffff:127.0.0.1:35542 - "GET /api/repos/gitlab/root/test-flake/builds?commit=860c014d5e7635ba9901c9e9bf9260e2b44a4446 HTTP/1.1" 200 OK gitlab # [ 248.915641] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:26.004Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"0e0ae333661ffd33d4acbe81","created_at":"2026-10-04T18:55:23.975Z","trace_propagation_headers":{"sentry-trace":"aa276fd2128d4c5b9500f0feb5c745d2-6a649ce8e2c349cd","baggage":"sentry-trace_id=aa276fd2128d4c5b9500f0feb5c745d2,sentry-release=Unknown"},"correlation_id":"01M444964XGQS1HFFSE980J6X3","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:23.977Z","job_size_bytes":23,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-0e0ae333661ffd33d4acbe81: start","job_status":"start","queue_duration_s":2.027335,"scheduling_latency_s":2.027372} gitlab # [ 248.957036] gitaly[916]: time="2026-10-04T18:55:26.046Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=653 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-10-04T18:55:56.036" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:26.036" grpc.time_ms=9.452 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.969564] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:26.053Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"0e0ae333661ffd33d4acbe81","created_at":"2026-10-04T18:55:23.975Z","trace_propagation_headers":{"sentry-trace":"aa276fd2128d4c5b9500f0feb5c745d2-6a649ce8e2c349cd","baggage":"sentry-trace_id=aa276fd2128d4c5b9500f0feb5c745d2,sentry-release=Unknown"},"correlation_id":"01M444964XGQS1HFFSE980J6X3","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:23.977Z","job_size_bytes":23,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-0e0ae333661ffd33d4acbe81: done: 0.049461 sec","job_status":"done","queue_duration_s":2.027335,"scheduling_latency_s":2.027372,"gitaly_calls":1,"gitaly_duration_s":0.011853,"redis_calls":16,"redis_duration_s":0.004837,"redis_read_bytes":52,"redis_write_bytes":2047,"redis_queues_calls":1,"redis_queues_duration_s":0.000344,"redis_queues_read_bytes":1,"redis_queues_write_bytes":978,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001687,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001859,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000947,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027672,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.049461,"completed_at":"2026-10-04T18:55:26.053Z","load_balancing_strategy":"primary","db_duration_s":0.002318,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 248.970289] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:26.055Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"3df312036b65b1194e3b0be3","created_at":"2026-10-04T18:55:24.342Z","trace_propagation_headers":{"sentry-trace":"aa276fd2128d4c5b9500f0feb5c745d2-6a649ce8e2c349cd","baggage":"sentry-trace_id=aa276fd2128d4c5b9500f0feb5c745d2,sentry-release=Unknown"},"correlation_id":"01M444964XGQS1HFFSE980J6X3","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f10e731963371f09eb9b14ba2a9bed95370be4ce3184aa20c8482510af0673fa","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:24.345Z","job_size_bytes":467,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-3df312036b65b1194e3b0be3: start","job_status":"start","queue_duration_s":1.709397,"scheduling_latency_s":1.709415} gitlab # [ 249.009584] gitaly[916]: time="2026-10-04T18:55:26.090Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T18:55:56.089" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:26.089" grpc.time_ms=0.598 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 249.048690] gitaly[916]: time="2026-10-04T18:55:26.138Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-04T18:55:36.038" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:55:26.128" grpc.time_ms=9.331 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 249.156885] gitaly[916]: time="2026-10-04T18:55:26.246Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-04T18:55:56.245" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:26.245" grpc.time_ms=0.735 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 249.171415] gitaly[916]: time="2026-10-04T18:55:26.260Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=18 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-10-04T18:55:56.258" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:26.258" grpc.time_ms=2.142 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 249.268934] gitaly[916]: time="2026-10-04T18:55:26.358Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T18:55:36.257" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:55:26.347" grpc.time_ms=10.985 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 249.720703] gitaly[916]: time="2026-10-04T18:55:26.810Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=968 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-04T18:55:56.792" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:26.792" grpc.time_ms=17.292 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 249.753501] gitaly[916]: time="2026-10-04T18:55:26.830Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444964XGQS1HFFSE980J6X3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-10-04T18:55:56.812" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:26.812" grpc.time_ms=17.367 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 249.777809] sidekiq-cluster[2522]: {"severity":"WARN","time":"2026-10-04T18:55:26.865Z","class":"CreatePipelineWorker","correlation_id":"01M444964XGQS1HFFSE980J6X3","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"2e64966c0f092bf33ef3bab01bb475f0f986e356","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"2e64966c0f092bf33ef3bab01bb475f0f986e356","ref":"refs/heads/master"},"retry":0} gitlab # [ 249.784607] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:26.873Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"3df312036b65b1194e3b0be3","created_at":"2026-10-04T18:55:24.342Z","trace_propagation_headers":{"sentry-trace":"aa276fd2128d4c5b9500f0feb5c745d2-6a649ce8e2c349cd","baggage":"sentry-trace_id=aa276fd2128d4c5b9500f0feb5c745d2,sentry-release=Unknown"},"correlation_id":"01M444964XGQS1HFFSE980J6X3","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f10e731963371f09eb9b14ba2a9bed95370be4ce3184aa20c8482510af0673fa","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:24.345Z","job_size_bytes":467,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-3df312036b65b1194e3b0be3: done: 0.818337 sec","job_status":"done","queue_duration_s":1.709397,"scheduling_latency_s":1.709415,"gitaly_calls":7,"gitaly_duration_s":0.120039,"redis_calls":51,"redis_duration_s":0.011827,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.00096,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.005035,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001759,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.00315,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000923,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.63091,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000331606,"duration_s":0.818337,"completed_at":"2026-10-04T18:55:26.873Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.051328,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 249.806073] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:26.875Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"b01bd334-9e0b-4a28-b2d6-a2d4e826f4f2\"}"],"class":"WebHookWorker","jid":"6b1eeae740bb0a78539bc60d","created_at":"2026-10-04T18:55:24.498Z","trace_propagation_headers":{"sentry-trace":"aa276fd2128d4c5b9500f0feb5c745d2-6a649ce8e2c349cd","baggage":"sentry-trace_id=aa276fd2128d4c5b9500f0feb5c745d2,sentry-release=Unknown"},"correlation_id":"01M444964XGQS1HFFSE980J6X3","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0ab801f554a9309fe6329f72bc6c8b149e8af27a8b9eb65ea67c254755addeca","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:24.504Z","job_size_bytes":1751,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-6b1eeae740bb0a78539bc60d: start","job_status":"start","queue_duration_s":2.371062,"scheduling_latency_s":2.371078} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=860c014d5e7635ba9901c9e9bf9260e2b44a4446' gitlab # [ 249.938835] nixbot[2645]: INFO: ::ffff:127.0.0.1:35564 - "GET /api/repos/gitlab/root/test-flake/builds?commit=860c014d5e7635ba9901c9e9bf9260e2b44a4446 HTTP/1.1" 200 OK gitlab # [ 249.939642] nixbot[2645]: INFO: ::ffff:127.0.0.1:35548 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=860c014d5e7635ba9901c9e9bf9260e2b44a4446', in 0.07 seconds) gitlab # [ 249.965091] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.053Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"b01bd334-9e0b-4a28-b2d6-a2d4e826f4f2\"}"],"class":"WebHookWorker","jid":"6b1eeae740bb0a78539bc60d","created_at":"2026-10-04T18:55:24.498Z","trace_propagation_headers":{"sentry-trace":"aa276fd2128d4c5b9500f0feb5c745d2-6a649ce8e2c349cd","baggage":"sentry-trace_id=aa276fd2128d4c5b9500f0feb5c745d2,sentry-release=Unknown"},"correlation_id":"01M444964XGQS1HFFSE980J6X3","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0ab801f554a9309fe6329f72bc6c8b149e8af27a8b9eb65ea67c254755addeca","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:24.504Z","job_size_bytes":1751,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-6b1eeae740bb0a78539bc60d: done: 0.178264 sec","job_status":"done","queue_duration_s":2.371062,"scheduling_latency_s":2.371078,"redis_calls":19,"redis_duration_s":0.0043879999999999995,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001187,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000561,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001895,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000745,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.056297099000005346,"cpu_s":0.069593,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000515708,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.178264,"completed_at":"2026-10-04T18:55:27.053Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002821,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.027471] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.056Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"10adcf4b0c84bf34d1a2cf99","created_at":"2026-10-04T18:55:25.021Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e9483c3dd5c947ed","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.025Z","job_size_bytes":23,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-10adcf4b0c84bf34d1a2cf99: start","job_status":"start","queue_duration_s":2.030623,"scheduling_latency_s":2.030654} gitlab # [ 250.031862] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.121Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"10adcf4b0c84bf34d1a2cf99","created_at":"2026-10-04T18:55:25.021Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e9483c3dd5c947ed","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.025Z","job_size_bytes":23,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-10adcf4b0c84bf34d1a2cf99: done: 0.065181 sec","job_status":"done","queue_duration_s":2.030623,"scheduling_latency_s":2.030654,"redis_calls":10,"redis_duration_s":0.002149,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000358,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001367,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000424,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045566,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000311213,"duration_s":0.065181,"completed_at":"2026-10-04T18:55:27.121Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013453,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 250.075724] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.122Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"ad643f6da9ec6f41a265ea8e","created_at":"2026-10-04T18:55:25.186Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e9483c3dd5c947ed","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.193Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-ad643f6da9ec6f41a265ea8e: start","job_status":"start","queue_duration_s":1.929278,"scheduling_latency_s":1.929292} gitlab # [ 250.076506] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.148Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"ad643f6da9ec6f41a265ea8e","created_at":"2026-10-04T18:55:25.186Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e9483c3dd5c947ed","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.193Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-ad643f6da9ec6f41a265ea8e: done: 0.02601 sec","job_status":"done","queue_duration_s":1.929278,"scheduling_latency_s":1.929292,"redis_calls":10,"redis_duration_s":0.001957,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000484,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001113,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00036,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016233,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000297245,"duration_s":0.02601,"completed_at":"2026-10-04T18:55:27.148Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004716,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.139576] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.149Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"8cd004b773a646b4c61ba61a","created_at":"2026-10-04T18:55:25.208Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e9483c3dd5c947ed","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.213Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-8cd004b773a646b4c61ba61a: start","job_status":"start","queue_duration_s":1.935798,"scheduling_latency_s":1.935814} gitlab # [ 250.140268] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.176Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"8cd004b773a646b4c61ba61a","created_at":"2026-10-04T18:55:25.208Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e9483c3dd5c947ed","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.213Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-8cd004b773a646b4c61ba61a: done: 0.026538 sec","job_status":"done","queue_duration_s":1.935798,"scheduling_latency_s":1.935814,"redis_calls":10,"redis_duration_s":0.0018839999999999998,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000363,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001163,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000358,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016174,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00030786,"duration_s":0.026538,"completed_at":"2026-10-04T18:55:27.176Z","load_balancing_strategy":"primary","db_duration_s":0.004985,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.181295] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.177Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"26717d59ac7934063dec199d","created_at":"2026-10-04T18:55:25.233Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e9483c3dd5c947ed","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.240Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-26717d59ac7934063dec199d: start","job_status":"start","queue_duration_s":1.937057,"scheduling_latency_s":1.937072} gitlab # [ 250.181879] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.204Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"26717d59ac7934063dec199d","created_at":"2026-10-04T18:55:25.233Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e9483c3dd5c947ed","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.240Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-26717d59ac7934063dec199d: done: 0.026697 sec","job_status":"done","queue_duration_s":1.937057,"scheduling_latency_s":1.937072,"redis_calls":13,"redis_duration_s":0.0027429999999999998,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000377,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001414,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000574,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000378,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016425,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000313448,"duration_s":0.026697,"completed_at":"2026-10-04T18:55:27.204Z","load_balancing_strategy":"primary","db_duration_s":0.004841,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.242283] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.205Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"67cd8084a32612b5a7702506","created_at":"2026-10-04T18:55:25.241Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e9483c3dd5c947ed","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.246Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-67cd8084a32612b5a7702506: start","job_status":"start","queue_duration_s":1.958604,"scheduling_latency_s":1.958619} gitlab # [ 250.243052] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.228Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"67cd8084a32612b5a7702506","created_at":"2026-10-04T18:55:25.241Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e9483c3dd5c947ed","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.246Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-67cd8084a32612b5a7702506: done: 0.022935 sec","job_status":"done","queue_duration_s":1.958604,"scheduling_latency_s":1.958619,"redis_calls":10,"redis_duration_s":0.001848,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000364,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001096,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000388,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014005,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000260927,"duration_s":0.022935,"completed_at":"2026-10-04T18:55:27.228Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004211,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.303961] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.229Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"ecb31e920c0f8cd0d4941549","created_at":"2026-10-04T18:55:25.247Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e9483c3dd5c947ed","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.253Z","job_size_bytes":19,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-ecb31e920c0f8cd0d4941549: start","job_status":"start","queue_duration_s":1.97649,"scheduling_latency_s":1.976506} gitlab # [ 250.304691] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.250Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"ecb31e920c0f8cd0d4941549","created_at":"2026-10-04T18:55:25.247Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e9483c3dd5c947ed","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.253Z","job_size_bytes":19,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-ecb31e920c0f8cd0d4941549: done: 0.020748 sec","job_status":"done","queue_duration_s":1.97649,"scheduling_latency_s":1.976506,"redis_calls":11,"redis_duration_s":0.002041,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000394,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.0013,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000347,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011689,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000245282,"duration_s":0.020748,"completed_at":"2026-10-04T18:55:27.250Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003886,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.350181] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.251Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"1e9edb04c23484da6b5f61e3","created_at":"2026-10-04T18:55:25.256Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e9483c3dd5c947ed","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.261Z","job_size_bytes":66,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-1e9edb04c23484da6b5f61e3: start","job_status":"start","queue_duration_s":1.990316,"scheduling_latency_s":1.99033} gitlab # [ 250.350815] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.343Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"1e9edb04c23484da6b5f61e3","created_at":"2026-10-04T18:55:25.256Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e9483c3dd5c947ed","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.261Z","job_size_bytes":66,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-1e9edb04c23484da6b5f61e3: done: 0.091622 sec","job_status":"done","queue_duration_s":1.990316,"scheduling_latency_s":1.99033,"redis_calls":10,"redis_duration_s":0.001891,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000519,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001013,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000359,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.07,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.068,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015905,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000235784,"duration_s":0.091622,"completed_at":"2026-10-04T18:55:27.343Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.07063,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.401025] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.344Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"7d6362b6d3df2ed67f541292","created_at":"2026-10-04T18:55:25.262Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e9483c3dd5c947ed","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.266Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-7d6362b6d3df2ed67f541292: start","job_status":"start","queue_duration_s":2.077996,"scheduling_latency_s":2.078011} gitlab # [ 250.401648] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.368Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"7d6362b6d3df2ed67f541292","created_at":"2026-10-04T18:55:25.262Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e9483c3dd5c947ed","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.266Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-7d6362b6d3df2ed67f541292: done: 0.024138 sec","job_status":"done","queue_duration_s":2.077996,"scheduling_latency_s":2.078011,"redis_calls":10,"redis_duration_s":0.0013050000000000002,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000253,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0008,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000252,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017158,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000215111,"duration_s":0.024138,"completed_at":"2026-10-04T18:55:27.368Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003043,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.458153] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.369Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"4153f513b65e7e0a87198fff","created_at":"2026-10-04T18:55:25.270Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e9483c3dd5c947ed","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.273Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-4153f513b65e7e0a87198fff: start","job_status":"start","queue_duration_s":2.096357,"scheduling_latency_s":2.096372} gitlab # [ 250.458817] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.392Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"4153f513b65e7e0a87198fff","created_at":"2026-10-04T18:55:25.270Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e9483c3dd5c947ed","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.273Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-4153f513b65e7e0a87198fff: done: 0.022386 sec","job_status":"done","queue_duration_s":2.096357,"scheduling_latency_s":2.096372,"redis_calls":10,"redis_duration_s":0.001343,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000254,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000835,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000254,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012848,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000274616,"duration_s":0.022386,"completed_at":"2026-10-04T18:55:27.391Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005576,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.483395] gitaly[916]: time="2026-10-04T18:55:27.512Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M444AKE6RSXQZZECFJRJ54ZZ diskcache=5483eb7e-9306-4ad4-80ab-e309ab84d16e grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-05T00:55:27.499" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:27.499" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.491611] gitaly[916]: time="2026-10-04T18:55:27.513Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=317 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444AKE6RSXQZZECFJRJ54ZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-05T00:55:27.499" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:27.499" grpc.time_ms=13.197 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.503529] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.392Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"c6e68c52ea748f2b3121209f","created_at":"2026-10-04T18:55:25.273Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e9483c3dd5c947ed","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.276Z","job_size_bytes":6,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-c6e68c52ea748f2b3121209f: start","job_status":"start","queue_duration_s":2.116241,"scheduling_latency_s":2.116256} gitlab # [ 250.552311] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.406Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"c6e68c52ea748f2b3121209f","created_at":"2026-10-04T18:55:25.273Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e9483c3dd5c947ed","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.276Z","job_size_bytes":6,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-c6e68c52ea748f2b3121209f: done: 0.013918 sec","job_status":"done","queue_duration_s":2.116241,"scheduling_latency_s":2.116256,"redis_calls":10,"redis_duration_s":0.001304,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000251,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000773,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00028,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008379,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000204216,"duration_s":0.013918,"completed_at":"2026-10-04T18:55:27.406Z","load_balancing_strategy":"primary","db_duration_s":0.001373,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.558827] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.407Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"08036cdad7786729dc47a86f","created_at":"2026-10-04T18:55:25.277Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e9483c3dd5c947ed","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.279Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-08036cdad7786729dc47a86f: start","job_status":"start","queue_duration_s":2.128037,"scheduling_latency_s":2.128049} gitlab # [ 250.572733] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.429Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"08036cdad7786729dc47a86f","created_at":"2026-10-04T18:55:25.277Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e9483c3dd5c947ed","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.279Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-08036cdad7786729dc47a86f: done: 0.021673 sec","job_status":"done","queue_duration_s":2.128037,"scheduling_latency_s":2.128049,"redis_calls":10,"redis_duration_s":0.001315,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000254,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0008,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000261,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013402,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000199467,"duration_s":0.021673,"completed_at":"2026-10-04T18:55:27.429Z","load_balancing_strategy":"primary","db_duration_s":0.003944,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.618203] gitaly[916]: time="2026-10-04T18:55:27.602Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444AKE6RSXQZZECFJRJ54ZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T18:55:57.540" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:27.601" grpc.time_ms=0.635 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.623113] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.430Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"5fa75377bfb554bb4340230c","created_at":"2026-10-04T18:55:25.284Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e9483c3dd5c947ed","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.287Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-5fa75377bfb554bb4340230c: start","job_status":"start","queue_duration_s":2.143191,"scheduling_latency_s":2.143204} gitlab # [ 250.637846] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.470Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"5fa75377bfb554bb4340230c","created_at":"2026-10-04T18:55:25.284Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e9483c3dd5c947ed","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.287Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-5fa75377bfb554bb4340230c: done: 0.039907 sec","job_status":"done","queue_duration_s":2.143191,"scheduling_latency_s":2.143204,"redis_calls":14,"redis_duration_s":0.002421,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000296,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001104,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001021,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025632,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000259809,"duration_s":0.039907,"completed_at":"2026-10-04T18:55:27.470Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007739,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.678779] gitaly[916]: time="2026-10-04T18:55:27.611Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444AKE6RSXQZZECFJRJ54ZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-04T18:55:37.084" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:55:27.605" grpc.time_ms=6.353 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.718780] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.471Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"6a518b7f7483a040500d0a40","created_at":"2026-10-04T18:55:25.411Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e9483c3dd5c947ed","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.413Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-6a518b7f7483a040500d0a40: start","job_status":"start","queue_duration_s":2.057702,"scheduling_latency_s":2.057738} gitlab # [ 250.719634] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.487Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"6a518b7f7483a040500d0a40","created_at":"2026-10-04T18:55:25.411Z","trace_propagation_headers":{"sentry-trace":"8ba6c869f2ff4ec6b0c3414ed7c3d67e-e9483c3dd5c947ed","baggage":"sentry-trace_id=8ba6c869f2ff4ec6b0c3414ed7c3d67e,sentry-release=Unknown"},"correlation_id":"01M4449B5N013PRGH0MEQR0G7W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.413Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-6a518b7f7483a040500d0a40: done: 0.016321 sec","job_status":"done","queue_duration_s":2.057702,"scheduling_latency_s":2.057738,"redis_calls":8,"redis_duration_s":0.001111,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000844,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000267,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011156,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016321,"completed_at":"2026-10-04T18:55:27.487Z","load_balancing_strategy":"primary","db_duration_s":0.002249,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 250.761063] gitaly[916]: time="2026-10-04T18:55:27.638Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444AKE6RSXQZZECFJRJ54ZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-04T18:55:57.636" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:27.637" grpc.time_ms=0.754 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.761714] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.488Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"2baa9c184ed9011fd0390d37","created_at":"2026-10-04T18:55:25.558Z","trace_propagation_headers":{"sentry-trace":"acf4e703890e4ea2ab1bed775db2704e-530de1dcf45a45b7","baggage":"sentry-trace_id=acf4e703890e4ea2ab1bed775db2704e,sentry-release=Unknown"},"correlation_id":"01M444AKE6RSXQZZECFJRJ54ZZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"e6511c706a85b57c59ff8161","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.561Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-2baa9c184ed9011fd0390d37: start","job_status":"start","queue_duration_s":1.927448,"scheduling_latency_s":1.927463} gitlab # [ 250.805340] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.523Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"2baa9c184ed9011fd0390d37","created_at":"2026-10-04T18:55:25.558Z","trace_propagation_headers":{"sentry-trace":"acf4e703890e4ea2ab1bed775db2704e-530de1dcf45a45b7","baggage":"sentry-trace_id=acf4e703890e4ea2ab1bed775db2704e,sentry-release=Unknown"},"correlation_id":"01M444AKE6RSXQZZECFJRJ54ZZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"e6511c706a85b57c59ff8161","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.561Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-2baa9c184ed9011fd0390d37: done: 0.035369 sec","job_status":"done","queue_duration_s":1.927448,"scheduling_latency_s":1.927463,"gitaly_calls":1,"gitaly_duration_s":0.015108,"redis_calls":12,"redis_duration_s":0.002129,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000833,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000374,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000922,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014406,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.024179888,"duration_s":0.035369,"completed_at":"2026-10-04T18:55:27.523Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003834,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.806768] gitaly[916]: time="2026-10-04T18:55:27.641Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=37 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444AKE6RSXQZZECFJRJ54ZZ error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-10-04T18:55:57.639" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:27.640" grpc.time_ms=1.798 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.852637] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:27.525Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"be9dc09746b6b18178f1f6f5","created_at":"2026-10-04T18:55:25.711Z","trace_propagation_headers":{"sentry-trace":"acf4e703890e4ea2ab1bed775db2704e-530de1dcf45a45b7","baggage":"sentry-trace_id=acf4e703890e4ea2ab1bed775db2704e,sentry-release=Unknown"},"correlation_id":"01M444AKE6RSXQZZECFJRJ54ZZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fcfd9d43467a0e2067b82ddb14a3d549a98489a2e8c2222e1495f5c23b1d68e1","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.712Z","job_size_bytes":467,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-be9dc09746b6b18178f1f6f5: start","job_status":"start","queue_duration_s":1.812367,"scheduling_latency_s":1.812381} gitlab # [ 250.853455] gitaly[916]: time="2026-10-04T18:55:27.708Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M444AKE6RSXQZZECFJRJ54ZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T18:55:37.182" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:55:27.702" grpc.time_ms=6.53 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.930743] gitaly[916]: time="2026-10-04T18:55:28.020Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444AS77ETAR28FKJH6A8CSR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:55:28.009" grpc.time_ms=10.657 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 250.931789] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:55:28 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 45 gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=860c014d5e7635ba9901c9e9bf9260e2b44a4446' gitlab # [ 250.989661] gitaly[916]: time="2026-10-04T18:55:28.079Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M444AS93MY875JQX08NJ76W9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:55:28.071" pid=916 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 250.990182] gitaly[916]: time="2026-10-04T18:55:28.079Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444AS93MY875JQX08NJ76W9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:55:28.071" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 250.991920] gitaly[916]: time="2026-10-04T18:55:28.081Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444AS93MY875JQX08NJ76W9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:55:28.071" grpc.time_ms=8.328 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 250.993455] gitlab-workhorse[826]: time="2026-10-04T18:55:28Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 250.994198] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:55:28 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 47 gitlab # [ 251.031880] gitaly[916]: time="2026-10-04T18:55:28.121Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=33 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=33 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=967 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444AKE6RSXQZZECFJRJ54ZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-04T18:55:58.077" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:28.078" grpc.time_ms=43.283 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 251.039698] nixbot[2645]: INFO: ::ffff:127.0.0.1:35576 - "GET /api/repos/gitlab/root/test-flake/builds?commit=860c014d5e7635ba9901c9e9bf9260e2b44a4446 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=860c014d5e7635ba9901c9e9bf9260e2b44a4446', in 0.10 seconds) gitlab # [ 251.050140] gitaly[916]: time="2026-10-04T18:55:28.139Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M444AKE6RSXQZZECFJRJ54ZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-10-04T18:55:58.124" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:28.124" grpc.time_ms=15.203 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 251.062350] sidekiq-cluster[2522]: {"severity":"WARN","time":"2026-10-04T18:55:28.151Z","class":"CreatePipelineWorker","correlation_id":"01M444AKE6RSXQZZECFJRJ54ZZ","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"2e64966c0f092bf33ef3bab01bb475f0f986e356","after":"860c014d5e7635ba9901c9e9bf9260e2b44a4446","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"860c014d5e7635ba9901c9e9bf9260e2b44a4446","ref":"refs/heads/master"},"retry":0} gitlab # [ 251.072597] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:28.161Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"be9dc09746b6b18178f1f6f5","created_at":"2026-10-04T18:55:25.711Z","trace_propagation_headers":{"sentry-trace":"acf4e703890e4ea2ab1bed775db2704e-530de1dcf45a45b7","baggage":"sentry-trace_id=acf4e703890e4ea2ab1bed775db2704e,sentry-release=Unknown"},"correlation_id":"01M444AKE6RSXQZZECFJRJ54ZZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fcfd9d43467a0e2067b82ddb14a3d549a98489a2e8c2222e1495f5c23b1d68e1","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.712Z","job_size_bytes":467,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-be9dc09746b6b18178f1f6f5: done: 0.636599 sec","job_status":"done","queue_duration_s":1.812367,"scheduling_latency_s":1.812381,"gitaly_calls":7,"gitaly_duration_s":0.191905,"redis_calls":33,"redis_duration_s":0.009969,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000378,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006982,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.001863,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000746,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.463531,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.636599,"completed_at":"2026-10-04T18:55:28.161Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014138,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 251.116600] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:28.205Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"841f5df6-ac25-4257-b257-706fc1a22cf0\"}"],"class":"WebHookWorker","jid":"b54d153ce6b9937515132972","created_at":"2026-10-04T18:55:25.723Z","trace_propagation_headers":{"sentry-trace":"acf4e703890e4ea2ab1bed775db2704e-530de1dcf45a45b7","baggage":"sentry-trace_id=acf4e703890e4ea2ab1bed775db2704e,sentry-release=Unknown"},"correlation_id":"01M444AKE6RSXQZZECFJRJ54ZZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:406c1b5d7ba5a080322aa6015ea6991f940c6d31dd20060eb7db5e1dfaeac74e","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.724Z","job_size_bytes":1749,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-b54d153ce6b9937515132972: start","job_status":"start","queue_duration_s":2.48102,"scheduling_latency_s":2.481049} gitlab # [ 251.140259] nixbot[2645]: INFO: ::ffff:127.0.0.1:35590 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 251.146028] gitaly[916]: time="2026-10-04T18:55:28.235Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M444ASAP6PVQMYKR04JQS9K5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:55:28.221" pid=916 remote_ip="2001:db8:1::1" request_sha=d54a086be4de3ea9860b15291ca1692d6a9441f4 response_bytes=337 span.kind=server system=grpc gitlab # [ 251.147980] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:28.237Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"841f5df6-ac25-4257-b257-706fc1a22cf0\"}"],"class":"WebHookWorker","jid":"b54d153ce6b9937515132972","created_at":"2026-10-04T18:55:25.723Z","trace_propagation_headers":{"sentry-trace":"acf4e703890e4ea2ab1bed775db2704e-530de1dcf45a45b7","baggage":"sentry-trace_id=acf4e703890e4ea2ab1bed775db2704e,sentry-release=Unknown"},"correlation_id":"01M444AKE6RSXQZZECFJRJ54ZZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:406c1b5d7ba5a080322aa6015ea6991f940c6d31dd20060eb7db5e1dfaeac74e","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.724Z","job_size_bytes":1749,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-b54d153ce6b9937515132972: done: 0.031905 sec","job_status":"done","queue_duration_s":2.48102,"scheduling_latency_s":2.481049,"redis_calls":14,"redis_duration_s":0.002058,"redis_read_bytes":25,"redis_write_bytes":4685,"redis_queues_calls":2,"redis_queues_duration_s":0.000433,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000913,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000712,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.007156776000016407,"cpu_s":0.016913,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.031905,"completed_at":"2026-10-04T18:55:28.237Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001355,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.188234] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:28.238Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"8848e33b49f43638ddd02dd7","created_at":"2026-10-04T18:55:25.847Z","trace_propagation_headers":{"sentry-trace":"aa276fd2128d4c5b9500f0feb5c745d2-4daf7c424f914958","baggage":"sentry-trace_id=aa276fd2128d4c5b9500f0feb5c745d2,sentry-release=Unknown"},"correlation_id":"01M444964XGQS1HFFSE980J6X3","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.851Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-8848e33b49f43638ddd02dd7: start","job_status":"start","queue_duration_s":2.38739,"scheduling_latency_s":2.387407} gitlab # [ 251.188831] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:28.271Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"8848e33b49f43638ddd02dd7","created_at":"2026-10-04T18:55:25.847Z","trace_propagation_headers":{"sentry-trace":"aa276fd2128d4c5b9500f0feb5c745d2-4daf7c424f914958","baggage":"sentry-trace_id=aa276fd2128d4c5b9500f0feb5c745d2,sentry-release=Unknown"},"correlation_id":"01M444964XGQS1HFFSE980J6X3","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:25.851Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-8848e33b49f43638ddd02dd7: done: 0.032636 sec","job_status":"done","queue_duration_s":2.38739,"scheduling_latency_s":2.387407,"redis_calls":19,"redis_duration_s":0.003432,"redis_read_bytes":21,"redis_write_bytes":4292,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000691,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000456,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2263,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001581,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000704,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021095,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0053699559999999995,"duration_s":0.032636,"completed_at":"2026-10-04T18:55:28.271Z","load_balancing_strategy":"primary","db_duration_s":0.00547,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.233645] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:28.272Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","6b1eeae740bb0a78539bc60d"],"class":"WebHooks::LogExecutionWorker","jid":"530c385914e2e698ec393a2a","created_at":"2026-10-04T18:55:27.033Z","trace_propagation_headers":{"sentry-trace":"aa276fd2128d4c5b9500f0feb5c745d2-952a7d2116d24c51","baggage":"sentry-trace_id=aa276fd2128d4c5b9500f0feb5c745d2,sentry-release=Unknown"},"correlation_id":"01M444964XGQS1HFFSE980J6X3","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b5c3adfe2d4184e366cb227ef8a855804d60a5dae36d14c1d0baee1a6909f83a","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:27.041Z","job_size_bytes":2593,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-530c385914e2e698ec393a2a: start","job_status":"start","queue_duration_s":1.230595,"scheduling_latency_s":1.230609} gitlab # [ 251.234484] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:28.305Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","6b1eeae740bb0a78539bc60d"],"class":"WebHooks::LogExecutionWorker","jid":"530c385914e2e698ec393a2a","created_at":"2026-10-04T18:55:27.033Z","trace_propagation_headers":{"sentry-trace":"aa276fd2128d4c5b9500f0feb5c745d2-952a7d2116d24c51","baggage":"sentry-trace_id=aa276fd2128d4c5b9500f0feb5c745d2,sentry-release=Unknown"},"correlation_id":"01M444964XGQS1HFFSE980J6X3","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b5c3adfe2d4184e366cb227ef8a855804d60a5dae36d14c1d0baee1a6909f83a","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:27.041Z","job_size_bytes":2593,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-530c385914e2e698ec393a2a: done: 0.032592 sec","job_status":"done","queue_duration_s":1.230595,"scheduling_latency_s":1.230609,"redis_calls":10,"redis_duration_s":0.00193,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000377,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00116,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000393,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020166,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000277689,"duration_s":0.032592,"completed_at":"2026-10-04T18:55:28.305Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007505,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.276837] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:28.306Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","b54d153ce6b9937515132972"],"class":"WebHooks::LogExecutionWorker","jid":"a815746a68dbc3d8b930b985","created_at":"2026-10-04T18:55:28.231Z","trace_propagation_headers":{"sentry-trace":"acf4e703890e4ea2ab1bed775db2704e-3ed317b9469747de","baggage":"sentry-trace_id=acf4e703890e4ea2ab1bed775db2704e,sentry-release=Unknown"},"correlation_id":"01M444AKE6RSXQZZECFJRJ54ZZ","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2e9386d2c880ebf33e6de3387c5515c4a8a9df2dd1608ee01d07d2d90b1ce72c","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:28.232Z","job_size_bytes":2592,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-a815746a68dbc3d8b930b985: start","job_status":"start","queue_duration_s":0.073405,"scheduling_latency_s":0.073419} gitlab # [ 251.319223] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:28.323Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","b54d153ce6b9937515132972"],"class":"WebHooks::LogExecutionWorker","jid":"a815746a68dbc3d8b930b985","created_at":"2026-10-04T18:55:28.231Z","trace_propagation_headers":{"sentry-trace":"acf4e703890e4ea2ab1bed775db2704e-3ed317b9469747de","baggage":"sentry-trace_id=acf4e703890e4ea2ab1bed775db2704e,sentry-release=Unknown"},"correlation_id":"01M444AKE6RSXQZZECFJRJ54ZZ","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2e9386d2c880ebf33e6de3387c5515c4a8a9df2dd1608ee01d07d2d90b1ce72c","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:28.232Z","job_size_bytes":2592,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-a815746a68dbc3d8b930b985: done: 0.017063 sec","job_status":"done","queue_duration_s":0.073405,"scheduling_latency_s":0.073419,"redis_calls":7,"redis_duration_s":0.001317,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000372,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010616,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017063,"completed_at":"2026-10-04T18:55:28.323Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003493,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.359272] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:28.324Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"33bc401be93720756619221e","created_at":"2026-10-04T18:55:28.261Z","trace_propagation_headers":{"sentry-trace":"aa276fd2128d4c5b9500f0feb5c745d2-6a48029c5b2b4e28","baggage":"sentry-trace_id=aa276fd2128d4c5b9500f0feb5c745d2,sentry-release=Unknown"},"correlation_id":"01M444964XGQS1HFFSE980J6X3","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:28.265Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-33bc401be93720756619221e: start","job_status":"start","queue_duration_s":0.05906,"scheduling_latency_s":0.059075} gitlab # [ 251.359860] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:28.388Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"33bc401be93720756619221e","created_at":"2026-10-04T18:55:28.261Z","trace_propagation_headers":{"sentry-trace":"aa276fd2128d4c5b9500f0feb5c745d2-6a48029c5b2b4e28","baggage":"sentry-trace_id=aa276fd2128d4c5b9500f0feb5c745d2,sentry-release=Unknown"},"correlation_id":"01M444964XGQS1HFFSE980J6X3","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:28.265Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-33bc401be93720756619221e: done: 0.063653 sec","job_status":"done","queue_duration_s":0.05906,"scheduling_latency_s":0.059075,"redis_calls":11,"redis_duration_s":0.002256,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000401,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001498,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000357,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042406,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026847,"duration_s":0.063653,"completed_at":"2026-10-04T18:55:28.388Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017721,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.417972] gitaly[916]: time="2026-10-04T18:55:28.235Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1193 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M444ASAP6PVQMYKR04JQS9K5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:55:28.221" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 251.418426] gitaly[916]: time="2026-10-04T18:55:28.508Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1193 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M444ASAP6PVQMYKR04JQS9K5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:55:28.221" grpc.time_ms=285.866 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 251.444227] gitlab-workhorse[826]: time="2026-10-04T18:55:28Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=337 gitlab # [ 251.445312] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:55:28 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 337 "" "git/2.55.0" 447 gitlab # [ 251.478828] nixbot[2645]: {"ts": "2026-10-04T18:55:28.568Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"} gitlab # [ 251.528724] gitaly[916]: time="2026-10-04T18:55:28.617Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444AST3TH8YWE2JKD952NN9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T18:55:58.516" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:28.617" grpc.time_ms=0.554 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 251.575736] gitaly[916]: time="2026-10-04T18:55:28.665Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444ASSJVJXT1YPVN2S15997 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:55:28.658" grpc.time_ms=7.127 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 251.576359] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:55:28 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 103 gitlab # [ 251.589149] gitaly[916]: time="2026-10-04T18:55:28.678Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M444AST3TH8YWE2JKD952NN9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T18:55:38.021" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:55:28.671" grpc.time_ms=6.443 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 251.637579] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:28.725Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"a08a7a687ce7a6750d70e804","created_at":"2026-10-04T18:55:28.717Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-33f9449225d741e6","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:28.724Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a08a7a687ce7a6750d70e804: start","job_status":"start","queue_duration_s":0.001374,"scheduling_latency_s":0.001406} gitlab # [ 251.647945] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:55:28 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/2e64966c0f092bf33ef3bab01bb475f0f986e356 HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 157 gitlab # [ 251.649165] nixbot[2645]: {"ts": "2026-10-04T18:55:28.738Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/2e64966c0f092bf33ef3bab01bb475f0f986e356 \"HTTP/1.1 201 Created\""} gitlab # [ 251.693197] gitaly[916]: time="2026-10-04T18:55:28.782Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M444ASWVTCZVJH66RY14EBHW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:55:28.773" pid=916 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 251.693861] gitaly[916]: time="2026-10-04T18:55:28.782Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444ASWVTCZVJH66RY14EBHW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:55:28.773" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 251.694759] gitaly[916]: time="2026-10-04T18:55:28.782Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444ASWVTCZVJH66RY14EBHW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:55:28.773" grpc.time_ms=9.249 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 251.698640] gitaly[916]: time="2026-10-04T18:55:28.788Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444ASZ6JW6WJJT60717S564 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T18:55:58.687" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:28.787" grpc.time_ms=0.5 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 251.721073] gitaly[916]: time="2026-10-04T18:55:28.809Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444AST3TH8YWE2JKD952NN9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:55:28.809" grpc.time_ms=0.31 pid=916 span.kind=server system=grpc gitlab # [ 251.731817] gitaly[916]: time="2026-10-04T18:55:28.821Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444ASZ6JW6WJJT60717S564 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T18:55:38.165" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:55:28.815" grpc.time_ms=5.415 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 251.745491] gitaly[916]: time="2026-10-04T18:55:28.835Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444AST3TH8YWE2JKD952NN9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:55:28.834" grpc.time_ms=0.288 pid=916 span.kind=server system=grpc gitlab # [ 251.765212] gitaly[916]: time="2026-10-04T18:55:28.854Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444AST3TH8YWE2JKD952NN9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:55:28.854" grpc.time_ms=0.496 pid=916 span.kind=server system=grpc gitlab # [ 251.775451] gitaly[916]: time="2026-10-04T18:55:28.865Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M444AST3TH8YWE2JKD952NN9 diskcache=bf2ca198-fd93-4c31-9497-1bb4d281adbe grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-04T18:55:58.779" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:55:28.780" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 251.776299] gitaly[916]: time="2026-10-04T18:55:28.865Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=73 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7048 command.oublock=0 command.real_time_ms=78 command.spawn_token_fork_ms=0 command.system_time_ms=43 command.user_time_ms=30 component=gitaly.UnaryServerInterceptor correlation_id=01M444AST3TH8YWE2JKD952NN9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-04T18:55:58.779" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:55:28.780" grpc.time_ms=84.684 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 251.810895] gitlab-workhorse[826]: time="2026-10-04T18:55:28Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 251.811922] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:55:28 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 233 gitlab # [ 251.812601] nixbot[2645]: {"ts": "2026-10-04T18:55:28.901Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/2e64966c0f092bf33ef3bab01bb475f0f986e356 \"HTTP/1.1 201 Created\""} gitlab # [ 251.813440] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:55:28 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/2e64966c0f092bf33ef3bab01bb475f0f986e356 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 160 gitlab # [ 251.832790] gitaly[916]: time="2026-10-04T18:55:28.922Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444AST3TH8YWE2JKD952NN9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-04T18:55:58.021" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:28.921" grpc.time_ms=0.613 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 251.867982] gitaly[916]: time="2026-10-04T18:55:28.957Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444AST3TH8YWE2JKD952NN9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T18:55:58.056" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:28.956" grpc.time_ms=0.658 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 251.893163] gitaly[916]: time="2026-10-04T18:55:28.982Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444AT4ANCFEVD6E4C1G6B40 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T18:55:58.881" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:28.981" grpc.time_ms=0.578 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 251.908090] gitaly[916]: time="2026-10-04T18:55:28.997Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444AT51Z167CYPJQGQHXZQV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:55:28.988" grpc.time_ms=9.257 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 251.909120] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:55:28 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 68 gitlab # [ 251.913870] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.002Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"7e9b32cc42fedb870918b2c1","created_at":"2026-10-04T18:55:29.000Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-8b2b33f53fa644b2","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"a08a7a687ce7a6750d70e804","job_size_bytes":3,"pid":2522,"job_status":"deduplicated","message":"PipelineProcessWorker JID-7e9b32cc42fedb870918b2c1: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60} gitlab # [ 251.915715] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.004Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"a08a7a687ce7a6750d70e804","created_at":"2026-10-04T18:55:28.717Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-33f9449225d741e6","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:28.724Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2522,"job_status":"rescheduled","message":"PipelineProcessWorker JID-a08a7a687ce7a6750d70e804: rescheduled"} gitlab # [ 251.920782] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.010Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"a08a7a687ce7a6750d70e804","created_at":"2026-10-04T18:55:28.717Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-33f9449225d741e6","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:28.724Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a08a7a687ce7a6750d70e804: done: 0.28436 sec","job_status":"done","queue_duration_s":0.001374,"scheduling_latency_s":0.001406,"gitaly_calls":3,"gitaly_duration_s":0.103416,"redis_calls":69,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.013868,"redis_read_bytes":226,"redis_write_bytes":18468,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000529,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001255,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.003334,"redis_queues_read_bytes":24,"redis_queues_write_bytes":12314,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.005979,"redis_queues_metadata_read_bytes":110,"redis_queues_metadata_write_bytes":3546,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000162,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000514,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001566,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.038,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":18,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.027,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.112209,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.21359698,"duration_s":0.28436,"completed_at":"2026-10-04T18:55:29.010Z","load_balancing_strategy":"primary","db_duration_s":0.039691,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 251.961622] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.011Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"903a7b9671fb89caff749caa","created_at":"2026-10-04T18:55:28.773Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-8b2b33f53fa644b2","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:28.775Z","job_size_bytes":23,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-903a7b9671fb89caff749caa: start","job_status":"start","queue_duration_s":0.23607,"scheduling_latency_s":0.23609} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=860c014d5e7635ba9901c9e9bf9260e2b44a4446' gitlab # [ 251.971409] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.040Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"903a7b9671fb89caff749caa","created_at":"2026-10-04T18:55:28.773Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-8b2b33f53fa644b2","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:28.775Z","job_size_bytes":23,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-903a7b9671fb89caff749caa: done: 0.029439 sec","job_status":"done","queue_duration_s":0.23607,"scheduling_latency_s":0.23609,"redis_calls":7,"redis_duration_s":0.001423,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00117,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000253,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019119,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029439,"completed_at":"2026-10-04T18:55:29.040Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006551,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 252.015140] gitaly[916]: time="2026-10-04T18:55:29.033Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444AT4ANCFEVD6E4C1G6B40 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T18:55:39.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:55:29.024" grpc.time_ms=9.317 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.022837] gitlab-workhorse[826]: time="2026-10-04T18:55:29Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=221 gitlab # [ 252.023680] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:55:29 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 221 "" "git/2.55.0" 64 gitlab # [ 252.024716] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.041Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"addb79b348dc1c642ab913a7","created_at":"2026-10-04T18:55:28.870Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-8b2b33f53fa644b2","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:28.872Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-addb79b348dc1c642ab913a7: start","job_status":"start","queue_duration_s":0.169178,"scheduling_latency_s":0.169193} gitlab # [ 252.069669] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.071Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"addb79b348dc1c642ab913a7","created_at":"2026-10-04T18:55:28.870Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-8b2b33f53fa644b2","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:28.872Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-addb79b348dc1c642ab913a7: done: 0.028771 sec","job_status":"done","queue_duration_s":0.169178,"scheduling_latency_s":0.169193,"redis_calls":7,"redis_duration_s":0.002573,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002141,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000432,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016466,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028771,"completed_at":"2026-10-04T18:55:29.070Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007527,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.112309] gitaly[916]: time="2026-10-04T18:55:29.065Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M444AT78WAYV0MQJK9E76HXX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:55:29.053" pid=916 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=221 span.kind=server system=grpc gitlab # [ 252.113286] nixbot[2645]: {"ts": "2026-10-04T18:55:29.140Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/2e64966c0f092bf33ef3bab01bb475f0f986e356 \"HTTP/1.1 201 Created\""} gitlab # [ 252.113707] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.072Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"f5c04244bcdff906859827d0","created_at":"2026-10-04T18:55:28.880Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-8b2b33f53fa644b2","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:28.883Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-f5c04244bcdff906859827d0: start","job_status":"start","queue_duration_s":0.189436,"scheduling_latency_s":0.189457} gitlab # [ 252.118171] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.109Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"f5c04244bcdff906859827d0","created_at":"2026-10-04T18:55:28.880Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-8b2b33f53fa644b2","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:28.883Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-f5c04244bcdff906859827d0: done: 0.036869 sec","job_status":"done","queue_duration_s":0.189436,"scheduling_latency_s":0.189457,"redis_calls":7,"redis_duration_s":0.015389,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.011996,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003393,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014149,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036869,"completed_at":"2026-10-04T18:55:29.109Z","load_balancing_strategy":"primary","db_duration_s":0.004542,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.160833] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:55:29 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/2e64966c0f092bf33ef3bab01bb475f0f986e356 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 219 gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=860c014d5e7635ba9901c9e9bf9260e2b44a4446', in 0.15 seconds) (finished: subtest: push delivers the webhook, in 49.92 seconds) gitlab # [ 252.161677] gitaly[916]: time="2026-10-04T18:55:29.065Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1173 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M444AT78WAYV0MQJK9E76HXX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:55:29.053" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc subtest: eval and build post commit statuses gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/860c014d5e7635ba9901c9e9bf9260e2b44a4446/statuses gitlab # [ 252.162491] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.110Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"2549385917841d5c6e4ae843","created_at":"2026-10-04T18:55:28.889Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-8b2b33f53fa644b2","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:28.891Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-2549385917841d5c6e4ae843: start","job_status":"start","queue_duration_s":0.219247,"scheduling_latency_s":0.219264} gitlab # [ 252.171413] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.133Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"2549385917841d5c6e4ae843","created_at":"2026-10-04T18:55:28.889Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-8b2b33f53fa644b2","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:28.891Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-2549385917841d5c6e4ae843: done: 0.022965 sec","job_status":"done","queue_duration_s":0.219247,"scheduling_latency_s":0.219264,"redis_calls":10,"redis_duration_s":0.001555,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000881,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00036,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000314,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014458,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022965,"completed_at":"2026-10-04T18:55:29.133Z","load_balancing_strategy":"primary","db_duration_s":0.004075,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.223527] gitaly[916]: time="2026-10-04T18:55:29.065Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1173 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M444AT78WAYV0MQJK9E76HXX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:55:29.053" grpc.time_ms=11.255 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 252.224198] nixbot[2645]: INFO: ::ffff:127.0.0.1:35598 - "GET /api/repos/gitlab/root/test-flake/builds?commit=860c014d5e7635ba9901c9e9bf9260e2b44a4446 HTTP/1.1" 200 OK gitlab # [ 252.230496] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.134Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"1ee5bb8a73a4c25b9fe4f429","created_at":"2026-10-04T18:55:28.891Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-8b2b33f53fa644b2","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:28.892Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-1ee5bb8a73a4c25b9fe4f429: start","job_status":"start","queue_duration_s":0.241911,"scheduling_latency_s":0.241936} gitlab # [ 252.281734] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.151Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"1ee5bb8a73a4c25b9fe4f429","created_at":"2026-10-04T18:55:28.891Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-8b2b33f53fa644b2","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:28.892Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-1ee5bb8a73a4c25b9fe4f429: done: 0.016642 sec","job_status":"done","queue_duration_s":0.241911,"scheduling_latency_s":0.241936,"redis_calls":7,"redis_duration_s":0.001093,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000799,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000294,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010922,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016642,"completed_at":"2026-10-04T18:55:29.151Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002779,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/860c014d5e7635ba9901c9e9bf9260e2b44a4446/statuses, in 0.13 seconds) gitlab # [ 252.285249] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.152Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"3b3c0d174594149b2ea0dafc","created_at":"2026-10-04T18:55:28.893Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-8b2b33f53fa644b2","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:28.895Z","job_size_bytes":19,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-3b3c0d174594149b2ea0dafc: start","job_status":"start","queue_duration_s":0.257888,"scheduling_latency_s":0.257903} gitlab # [ 252.328962] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.168Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"3b3c0d174594149b2ea0dafc","created_at":"2026-10-04T18:55:28.893Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-8b2b33f53fa644b2","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:28.895Z","job_size_bytes":19,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-3b3c0d174594149b2ea0dafc: done: 0.015776 sec","job_status":"done","queue_duration_s":0.257888,"scheduling_latency_s":0.257903,"redis_calls":8,"redis_duration_s":0.0015660000000000001,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001307,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000259,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009617,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015776,"completed_at":"2026-10-04T18:55:29.168Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002194,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.338832] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:55:29 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 157 gitlab # [ 252.340713] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:55:29 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/860c014d5e7635ba9901c9e9bf9260e2b44a4446/statuses HTTP/1.1" 200 2 "" "curl/8.21.0" 105 gitlab # [ 252.343076] gitaly[916]: time="2026-10-04T18:55:29.323Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=96 command.spawn_token_fork_ms=91 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M444ATCE59S89SAXR387Z1R6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:55:29.225" grpc.time_ms=97.94 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 252.383779] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.169Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"47144e39844e2820bf7d82fe","created_at":"2026-10-04T18:55:28.895Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-8b2b33f53fa644b2","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:28.896Z","job_size_bytes":66,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-47144e39844e2820bf7d82fe: start","job_status":"start","queue_duration_s":0.272793,"scheduling_latency_s":0.272807} gitlab # [ 252.384636] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.184Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"47144e39844e2820bf7d82fe","created_at":"2026-10-04T18:55:28.895Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-8b2b33f53fa644b2","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:28.896Z","job_size_bytes":66,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-47144e39844e2820bf7d82fe: done: 0.014859 sec","job_status":"done","queue_duration_s":0.272793,"scheduling_latency_s":0.272807,"redis_calls":7,"redis_duration_s":0.001101,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000795,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000306,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009549,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014859,"completed_at":"2026-10-04T18:55:29.184Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002037,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.430405] gitlab-workhorse[826]: time="2026-10-04T18:55:29Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 252.431110] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:55:29 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 120 gitlab # [ 252.432510] gitaly[916]: time="2026-10-04T18:55:29.363Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444ATE3K12C2WKMCTG9N28S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T18:55:59.261" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:29.362" grpc.time_ms=0.601 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.433488] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.185Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"dad5f9ad47240c29ac45d4aa","created_at":"2026-10-04T18:55:28.897Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-8b2b33f53fa644b2","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:28.898Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-dad5f9ad47240c29ac45d4aa: start","job_status":"start","queue_duration_s":0.287434,"scheduling_latency_s":0.28745} gitlab # [ 252.441919] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.203Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"dad5f9ad47240c29ac45d4aa","created_at":"2026-10-04T18:55:28.897Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-8b2b33f53fa644b2","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:28.898Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-dad5f9ad47240c29ac45d4aa: done: 0.018156 sec","job_status":"done","queue_duration_s":0.287434,"scheduling_latency_s":0.28745,"redis_calls":7,"redis_duration_s":0.001093,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000817,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000276,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010412,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018156,"completed_at":"2026-10-04T18:55:29.203Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005109,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.483661] gitaly[916]: time="2026-10-04T18:55:29.379Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444ATGTSJWNE6F66033R6BF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T18:55:59.077" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:29.378" grpc.time_ms=0.651 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.521683] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.205Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"a771caee2d4494dbc8292abd","created_at":"2026-10-04T18:55:28.898Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-8b2b33f53fa644b2","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:28.900Z","job_size_bytes":6,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-a771caee2d4494dbc8292abd: start","job_status":"start","queue_duration_s":0.304993,"scheduling_latency_s":0.305008} gitlab # [ 252.522592] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.214Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"a771caee2d4494dbc8292abd","created_at":"2026-10-04T18:55:28.898Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-8b2b33f53fa644b2","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:28.900Z","job_size_bytes":6,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-a771caee2d4494dbc8292abd: done: 0.009506 sec","job_status":"done","queue_duration_s":0.304993,"scheduling_latency_s":0.305008,"redis_calls":7,"redis_duration_s":0.001301,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001029,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000272,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005893,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.009506,"completed_at":"2026-10-04T18:55:29.214Z","load_balancing_strategy":"primary","db_duration_s":0.000285,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.567669] gitaly[916]: time="2026-10-04T18:55:29.381Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444ATE3K12C2WKMCTG9N28S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-10-04T18:55:59.273" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:55:29.373" grpc.time_ms=7.762 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.568076] nixbot[2645]: {"ts": "2026-10-04T18:55:29.624Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/860c014d5e7635ba9901c9e9bf9260e2b44a4446 \"HTTP/1.1 201 Created\""} gitlab # [ 252.569494] nixbot[2645]: {"ts": "2026-10-04T18:55:29.633Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-860c014d5e76-4"} gitlab # [ 252.570630] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.215Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"26e0520767bcae480ec40bb9","created_at":"2026-10-04T18:55:28.909Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-8b2b33f53fa644b2","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:28.912Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-26e0520767bcae480ec40bb9: start","job_status":"start","queue_duration_s":0.303425,"scheduling_latency_s":0.303439} gitlab # [ 252.575026] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.244Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"26e0520767bcae480ec40bb9","created_at":"2026-10-04T18:55:28.909Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-8b2b33f53fa644b2","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:28.912Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-26e0520767bcae480ec40bb9: done: 0.028558 sec","job_status":"done","queue_duration_s":0.303425,"scheduling_latency_s":0.303439,"redis_calls":7,"redis_duration_s":0.004977,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0046,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000377,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013833,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028558,"completed_at":"2026-10-04T18:55:29.244Z","load_balancing_strategy":"primary","db_duration_s":0.005346,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.614121] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:55:29 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/860c014d5e7635ba9901c9e9bf9260e2b44a4446 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 404 gitlab # [ 252.614777] gitaly[916]: time="2026-10-04T18:55:29.402Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444ATE3K12C2WKMCTG9N28S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T18:55:39.273" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:55:29.393" grpc.time_ms=8.249 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.663044] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.248Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"58418ffb673276e017d47b7d","created_at":"2026-10-04T18:55:29.004Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-8b2b33f53fa644b2","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.006Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-58418ffb673276e017d47b7d: start","job_status":"start","queue_duration_s":0.241394,"scheduling_latency_s":0.241415} gitlab # [ 252.663661] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.611Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"58418ffb673276e017d47b7d","created_at":"2026-10-04T18:55:29.004Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-8b2b33f53fa644b2","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.006Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2522,"job_status":"rescheduled","message":"PipelineProcessWorker JID-58418ffb673276e017d47b7d: rescheduled"} gitlab # [ 252.664481] nixbot[2645]: {"ts": "2026-10-04T18:55:29.733Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-860c014d5e76-4", "argv0": "bwrap"} gitlab # [ 252.664957] gitaly[916]: time="2026-10-04T18:55:29.437Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M444ATE3K12C2WKMCTG9N28S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-04T18:55:39.006" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:55:29.436" grpc.time_ms=0.201 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.665595] gitaly[916]: time="2026-10-04T18:55:29.445Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M444ATHDZR63VQA01FF4XEND grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:55:29.435" pid=916 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 252.705789] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.618Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"58418ffb673276e017d47b7d","created_at":"2026-10-04T18:55:29.004Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-8b2b33f53fa644b2","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.006Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-58418ffb673276e017d47b7d: done: 0.370089 sec","job_status":"done","queue_duration_s":0.241394,"scheduling_latency_s":0.241415,"gitaly_calls":3,"gitaly_duration_s":0.241682,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.010387,"redis_read_bytes":148,"redis_write_bytes":18013,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.00046,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.00064,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.002405,"redis_queues_read_bytes":26,"redis_queues_write_bytes":12312,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.00462,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000196,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000354,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001252,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.008,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.008,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.088719,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.335106836,"duration_s":0.370089,"completed_at":"2026-10-04T18:55:29.618Z","load_balancing_strategy":"primary","db_duration_s":0.030572,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 252.758653] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.619Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"488f375fb09c98eb175d718f","created_at":"2026-10-04T18:55:29.303Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-e00197410563454a","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.305Z","job_size_bytes":23,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-488f375fb09c98eb175d718f: start","job_status":"start","queue_duration_s":0.313596,"scheduling_latency_s":0.313612} gitlab # [ 252.759554] gitaly[916]: time="2026-10-04T18:55:29.445Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M444ATHDZR63VQA01FF4XEND grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:55:29.435" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 252.760466] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.645Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"488f375fb09c98eb175d718f","created_at":"2026-10-04T18:55:29.303Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-e00197410563454a","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.305Z","job_size_bytes":23,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-488f375fb09c98eb175d718f: done: 0.026703 sec","job_status":"done","queue_duration_s":0.313596,"scheduling_latency_s":0.313612,"redis_calls":7,"redis_duration_s":0.0009339999999999999,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000695,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000239,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018797,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026703,"completed_at":"2026-10-04T18:55:29.645Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004973,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 252.787585] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.646Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"52208b360d7f8e6270f97024","created_at":"2026-10-04T18:55:29.550Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-e00197410563454a","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.552Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-52208b360d7f8e6270f97024: start","job_status":"start","queue_duration_s":0.093864,"scheduling_latency_s":0.093882} gitlab # [ 252.836865] gitaly[916]: time="2026-10-04T18:55:29.445Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M444ATHDZR63VQA01FF4XEND grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T18:55:29.435" grpc.time_ms=9.937 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 252.837705] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.667Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"52208b360d7f8e6270f97024","created_at":"2026-10-04T18:55:29.550Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-e00197410563454a","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.552Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-52208b360d7f8e6270f97024: done: 0.020584 sec","job_status":"done","queue_duration_s":0.093864,"scheduling_latency_s":0.093882,"redis_calls":7,"redis_duration_s":0.001003,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000698,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000305,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011847,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020584,"completed_at":"2026-10-04T18:55:29.667Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00551,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.878998] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.668Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"cc399f451aaf75dcf26583b4","created_at":"2026-10-04T18:55:29.559Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-e00197410563454a","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.561Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-cc399f451aaf75dcf26583b4: start","job_status":"start","queue_duration_s":0.107247,"scheduling_latency_s":0.107288} gitlab # [ 252.879767] gitaly[916]: time="2026-10-04T18:55:29.447Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=620 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444ATE3K12C2WKMCTG9N28S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-04T18:55:39.010" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:55:29.440" grpc.time_ms=6.43 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.883770] gitaly[916]: time="2026-10-04T18:55:29.481Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444AST3TH8YWE2JKD952NN9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:55:29.481" grpc.time_ms=0.282 pid=916 span.kind=server system=grpc gitlab # [ 252.892741] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.689Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"cc399f451aaf75dcf26583b4","created_at":"2026-10-04T18:55:29.559Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-e00197410563454a","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.561Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-cc399f451aaf75dcf26583b4: done: 0.0208 sec","job_status":"done","queue_duration_s":0.107247,"scheduling_latency_s":0.107288,"redis_calls":7,"redis_duration_s":0.0009320000000000001,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00067,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000262,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012481,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0208,"completed_at":"2026-10-04T18:55:29.689Z","load_balancing_strategy":"primary","db_duration_s":0.005412,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.933376] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.690Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"13e72ace27fb37500f6ae7cb","created_at":"2026-10-04T18:55:29.560Z","trace_propagation_headers":{"sentry-trace":"63638d8da47d47ad999827e21b75e9a7-4c5a9bfc404d4161","baggage":"sentry-trace_id=63638d8da47d47ad999827e21b75e9a7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444ATE3K12C2WKMCTG9N28S","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.563Z","job_size_bytes":65,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-13e72ace27fb37500f6ae7cb: start","job_status":"start","queue_duration_s":0.126448,"scheduling_latency_s":0.126465} gitlab # [ 252.946352] gitaly[916]: time="2026-10-04T18:55:29.501Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444ATE3K12C2WKMCTG9N28S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:55:29.501" grpc.time_ms=0.238 pid=916 span.kind=server system=grpc gitlab # [ 252.955265] gitaly[916]: time="2026-10-04T18:55:29.507Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444AST3TH8YWE2JKD952NN9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:55:29.506" grpc.time_ms=0.221 pid=916 span.kind=server system=grpc gitlab # [ 252.964313] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.714Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"13e72ace27fb37500f6ae7cb","created_at":"2026-10-04T18:55:29.560Z","trace_propagation_headers":{"sentry-trace":"63638d8da47d47ad999827e21b75e9a7-4c5a9bfc404d4161","baggage":"sentry-trace_id=63638d8da47d47ad999827e21b75e9a7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444ATE3K12C2WKMCTG9N28S","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.563Z","job_size_bytes":65,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-13e72ace27fb37500f6ae7cb: done: 0.024241 sec","job_status":"done","queue_duration_s":0.126448,"scheduling_latency_s":0.126465,"redis_calls":7,"redis_duration_s":0.001183,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00091,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000273,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014251,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024241,"completed_at":"2026-10-04T18:55:29.714Z","load_balancing_strategy":"primary","db_duration_s":0.00661,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 253.031116] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.715Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"56f5191bbd9488bf14a437a5","created_at":"2026-10-04T18:55:29.564Z","trace_propagation_headers":{"sentry-trace":"63638d8da47d47ad999827e21b75e9a7-4c5a9bfc404d4161","baggage":"sentry-trace_id=63638d8da47d47ad999827e21b75e9a7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444ATE3K12C2WKMCTG9N28S","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.566Z","job_size_bytes":65,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-56f5191bbd9488bf14a437a5: start","job_status":"start","queue_duration_s":0.14948,"scheduling_latency_s":0.149503} gitlab # [ 253.031995] gitaly[916]: time="2026-10-04T18:55:29.533Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444AST3TH8YWE2JKD952NN9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:55:29.533" grpc.time_ms=0.239 pid=916 span.kind=server system=grpc gitlab # [ 253.032662] gitaly[916]: time="2026-10-04T18:55:29.535Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444ATE3K12C2WKMCTG9N28S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:55:29.534" grpc.time_ms=0.372 pid=916 span.kind=server system=grpc gitlab # [ 253.037519] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.753Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"56f5191bbd9488bf14a437a5","created_at":"2026-10-04T18:55:29.564Z","trace_propagation_headers":{"sentry-trace":"63638d8da47d47ad999827e21b75e9a7-4c5a9bfc404d4161","baggage":"sentry-trace_id=63638d8da47d47ad999827e21b75e9a7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444ATE3K12C2WKMCTG9N28S","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.566Z","job_size_bytes":65,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-56f5191bbd9488bf14a437a5: done: 0.037243 sec","job_status":"done","queue_duration_s":0.14948,"scheduling_latency_s":0.149503,"redis_calls":10,"redis_duration_s":0.001471,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000765,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000706,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023585,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037243,"completed_at":"2026-10-04T18:55:29.753Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009072,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.085187] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.754Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"1fb2cb65c80a8fadbc8084ab","created_at":"2026-10-04T18:55:29.567Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-e00197410563454a","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.568Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-1fb2cb65c80a8fadbc8084ab: start","job_status":"start","queue_duration_s":0.186031,"scheduling_latency_s":0.186049} gitlab # [ 253.128757] gitaly[916]: time="2026-10-04T18:55:29.538Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M444ATE3K12C2WKMCTG9N28S forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T18:55:39.018" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:55:29.448" operations="[repack_refs]" pid=916 ref=refs/keep-around/860c014d5e7635ba9901c9e9bf9260e2b44a4446 remote_ip="2001:db8:1::1" revision=860c014d5e7635ba9901c9e9bf9260e2b44a4446 span.kind=server system=grpc username=root gitlab # [ 253.129696] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.773Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"1fb2cb65c80a8fadbc8084ab","created_at":"2026-10-04T18:55:29.567Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-e00197410563454a","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.568Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-1fb2cb65c80a8fadbc8084ab: done: 0.019612 sec","job_status":"done","queue_duration_s":0.186031,"scheduling_latency_s":0.186049,"redis_calls":10,"redis_duration_s":0.00125,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000657,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000348,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000245,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013234,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019612,"completed_at":"2026-10-04T18:55:29.773Z","load_balancing_strategy":"primary","db_duration_s":0.003198,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.140648] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.774Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"77614bcf53980d322781e855","created_at":"2026-10-04T18:55:29.568Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-e00197410563454a","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.569Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-77614bcf53980d322781e855: start","job_status":"start","queue_duration_s":0.205193,"scheduling_latency_s":0.205208} gitlab # [ 253.183108] gitaly[916]: time="2026-10-04T18:55:29.539Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M444ATE3K12C2WKMCTG9N28S grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T18:55:39.018" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:55:29.448" pid=916 ref=refs/keep-around/860c014d5e7635ba9901c9e9bf9260e2b44a4446 remote_ip="2001:db8:1::1" repository_info="{false {6 703 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-10-04 18:54:28.936176084 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" revision=860c014d5e7635ba9901c9e9bf9260e2b44a4446 span.kind=server system=grpc username=root gitlab # [ 253.183833] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.790Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"77614bcf53980d322781e855","created_at":"2026-10-04T18:55:29.568Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-e00197410563454a","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.569Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-77614bcf53980d322781e855: done: 0.015531 sec","job_status":"done","queue_duration_s":0.205193,"scheduling_latency_s":0.205208,"redis_calls":7,"redis_duration_s":0.000881,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000668,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000213,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010267,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015531,"completed_at":"2026-10-04T18:55:29.790Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002599,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.227281] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.791Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"669a6e760dbbc4635b4bbbcf","created_at":"2026-10-04T18:55:29.569Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-e00197410563454a","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.571Z","job_size_bytes":19,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-669a6e760dbbc4635b4bbbcf: start","job_status":"start","queue_duration_s":0.219884,"scheduling_latency_s":0.219898} gitlab # [ 253.227988] gitaly[916]: time="2026-10-04T18:55:29.544Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M444AST3TH8YWE2JKD952NN9 diskcache=f34952ab-222d-49d5-9a6c-e2b470a71aef grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-04T18:55:59.009" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:55:29.309" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 253.230461] gitaly[916]: time="2026-10-04T18:55:29.544Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=74 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=6979 command.oublock=0 command.real_time_ms=225 command.spawn_token_fork_ms=2 command.system_time_ms=46 command.user_time_ms=28 component=gitaly.UnaryServerInterceptor correlation_id=01M444AST3TH8YWE2JKD952NN9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-04T18:55:59.009" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:55:29.309" grpc.time_ms=234.726 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 253.278951] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.804Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"669a6e760dbbc4635b4bbbcf","created_at":"2026-10-04T18:55:29.569Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-e00197410563454a","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.571Z","job_size_bytes":19,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-669a6e760dbbc4635b4bbbcf: done: 0.013788 sec","job_status":"done","queue_duration_s":0.219884,"scheduling_latency_s":0.219898,"redis_calls":8,"redis_duration_s":0.000954,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000717,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000237,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008956,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013788,"completed_at":"2026-10-04T18:55:29.804Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002061,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/860c014d5e7635ba9901c9e9bf9260e2b44a4446/statuses gitlab # [ 253.281154] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.805Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"0ed2d4163ca20e37fb8027df","created_at":"2026-10-04T18:55:29.571Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-e00197410563454a","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.572Z","job_size_bytes":66,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-0ed2d4163ca20e37fb8027df: start","job_status":"start","queue_duration_s":0.233002,"scheduling_latency_s":0.233015} gitlab # [ 253.325656] gitaly[916]: time="2026-10-04T18:55:29.546Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M444ATE3K12C2WKMCTG9N28S diskcache=e9f1958c-4e1c-4e3e-b103-074ca4d64e0a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T18:55:39.018" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:55:29.448" pid=916 ref=refs/keep-around/860c014d5e7635ba9901c9e9bf9260e2b44a4446 remote_ip="2001:db8:1::1" revision=860c014d5e7635ba9901c9e9bf9260e2b44a4446 span.kind=server system=grpc username=root gitlab # [ 253.326787] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.820Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"0ed2d4163ca20e37fb8027df","created_at":"2026-10-04T18:55:29.571Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-e00197410563454a","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.572Z","job_size_bytes":66,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-0ed2d4163ca20e37fb8027df: done: 0.014751 sec","job_status":"done","queue_duration_s":0.233002,"scheduling_latency_s":0.233015,"redis_calls":7,"redis_duration_s":0.001167,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000778,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000389,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008937,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014751,"completed_at":"2026-10-04T18:55:29.820Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002067,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.347667] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.821Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"249d9caee8c583eb5face4b5","created_at":"2026-10-04T18:55:29.573Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-e00197410563454a","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.574Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-249d9caee8c583eb5face4b5: start","job_status":"start","queue_duration_s":0.247234,"scheduling_latency_s":0.247247} gitlab # [ 253.362702] gitaly[916]: time="2026-10-04T18:55:29.546Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=50 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5496 command.oublock=8 command.real_time_ms=90 command.spawn_token_fork_ms=0 command.system_time_ms=34 command.user_time_ms=16 component=gitaly.UnaryServerInterceptor correlation_id=01M444ATE3K12C2WKMCTG9N28S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T18:55:39.018" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:55:29.448" grpc.time_ms=97.691 pid=916 ref=refs/keep-around/860c014d5e7635ba9901c9e9bf9260e2b44a4446 remote_ip="2001:db8:1::1" revision=860c014d5e7635ba9901c9e9bf9260e2b44a4446 span.kind=server system=grpc username=root gitlab # [ 253.376205] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.839Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"249d9caee8c583eb5face4b5","created_at":"2026-10-04T18:55:29.573Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-e00197410563454a","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.574Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-249d9caee8c583eb5face4b5: done: 0.018084 sec","job_status":"done","queue_duration_s":0.247234,"scheduling_latency_s":0.247247,"redis_calls":7,"redis_duration_s":0.0011380000000000001,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000883,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000255,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00964,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018084,"completed_at":"2026-10-04T18:55:29.839Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005653,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.440365] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.840Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"442aa640077ebba21991d6db","created_at":"2026-10-04T18:55:29.574Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-e00197410563454a","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.575Z","job_size_bytes":6,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-442aa640077ebba21991d6db: start","job_status":"start","queue_duration_s":0.265067,"scheduling_latency_s":0.265081} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/860c014d5e7635ba9901c9e9bf9260e2b44a4446/statuses, in 0.14 seconds) gitlab # [ 253.441025] gitaly[916]: time="2026-10-04T18:55:29.551Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444ATE3K12C2WKMCTG9N28S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T18:55:59.450" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:29.551" grpc.time_ms=0.461 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 253.441630] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.848Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"442aa640077ebba21991d6db","created_at":"2026-10-04T18:55:29.574Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-e00197410563454a","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.575Z","job_size_bytes":6,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-442aa640077ebba21991d6db: done: 0.008448 sec","job_status":"done","queue_duration_s":0.265067,"scheduling_latency_s":0.265081,"redis_calls":7,"redis_duration_s":0.0009419999999999999,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000676,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000266,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005338,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008448,"completed_at":"2026-10-04T18:55:29.848Z","load_balancing_strategy":"primary","db_duration_s":0.00028,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.509188] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.849Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"2c16bf66a16371c215389c6a","created_at":"2026-10-04T18:55:29.575Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-e00197410563454a","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.576Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-2c16bf66a16371c215389c6a: start","job_status":"start","queue_duration_s":0.273214,"scheduling_latency_s":0.273227} gitlab # [ 253.509926] gitaly[916]: time="2026-10-04T18:55:29.555Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M444ATE3K12C2WKMCTG9N28S grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T18:55:39.018" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:55:29.448" pid=916 ref=refs/keep-around/860c014d5e7635ba9901c9e9bf9260e2b44a4446 remote_ip="2001:db8:1::1" revision=860c014d5e7635ba9901c9e9bf9260e2b44a4446 span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root gitlab # [ 253.510523] gitaly[916]: time="2026-10-04T18:55:29.555Z" level=info msg="optimized repository" clean_stale_data_time=14.97425ms clean_work_trees_time="106.438µs" commit_graph_time="3.631µs" correlation_id=01M444ATE3K12C2WKMCTG9N28S gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="4.749µs" pid=916 prune_time="3.352µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="4.749µs" storage_name=default system=housekeeping total_time=15.197182ms gitlab # [ 253.511566] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:55:30 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/860c014d5e7635ba9901c9e9bf9260e2b44a4446/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 83 gitlab # [ 253.513040] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.863Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"2c16bf66a16371c215389c6a","created_at":"2026-10-04T18:55:29.575Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-e00197410563454a","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.576Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-2c16bf66a16371c215389c6a: done: 0.013335 sec","job_status":"done","queue_duration_s":0.273214,"scheduling_latency_s":0.273227,"redis_calls":7,"redis_duration_s":0.000851,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000583,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000268,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008879,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013335,"completed_at":"2026-10-04T18:55:29.862Z","load_balancing_strategy":"primary","db_duration_s":0.001968,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.583853] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.863Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"613776e71f6c3989ce70b474","created_at":"2026-10-04T18:55:29.609Z","trace_propagation_headers":{"sentry-trace":"63638d8da47d47ad999827e21b75e9a7-4c5a9bfc404d4161","baggage":"sentry-trace_id=63638d8da47d47ad999827e21b75e9a7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444ATE3K12C2WKMCTG9N28S","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.612Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-613776e71f6c3989ce70b474: start","job_status":"start","queue_duration_s":0.251638,"scheduling_latency_s":0.251651} gitlab # [ 253.585072] gitaly[916]: time="2026-10-04T18:55:29.555Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M444ATE3K12C2WKMCTG9N28S grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T18:55:39.018" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T18:55:29.448" pid=916 ref=refs/keep-around/860c014d5e7635ba9901c9e9bf9260e2b44a4446 remote_ip="2001:db8:1::1" revision=860c014d5e7635ba9901c9e9bf9260e2b44a4446 span.kind=server system=grpc username=root gitlab # [ 253.585757] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.942Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"613776e71f6c3989ce70b474","created_at":"2026-10-04T18:55:29.609Z","trace_propagation_headers":{"sentry-trace":"63638d8da47d47ad999827e21b75e9a7-4c5a9bfc404d4161","baggage":"sentry-trace_id=63638d8da47d47ad999827e21b75e9a7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M444ATE3K12C2WKMCTG9N28S","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.612Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-613776e71f6c3989ce70b474: done: 0.078402 sec","job_status":"done","queue_duration_s":0.251638,"scheduling_latency_s":0.251651,"gitaly_calls":2,"gitaly_duration_s":0.005258,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.00412,"redis_read_bytes":79,"redis_write_bytes":5122,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000555,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000782,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000162,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000984,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00014,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000338,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000604,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.054493,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.060255144,"duration_s":0.078402,"completed_at":"2026-10-04T18:55:29.942Z","load_balancing_strategy":"primary","db_duration_s":0.018066,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 253.622810] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.943Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"5e2d83e28cc2bb9a4c8cf670","created_at":"2026-10-04T18:55:29.611Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-e00197410563454a","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.614Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5e2d83e28cc2bb9a4c8cf670: start","job_status":"start","queue_duration_s":0.328921,"scheduling_latency_s":0.328941} gitlab # [ 253.672764] gitaly[916]: time="2026-10-04T18:55:29.584Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444AST3TH8YWE2JKD952NN9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-04T18:55:59.283" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:29.583" grpc.time_ms=0.619 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 253.673623] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.960Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"5e2d83e28cc2bb9a4c8cf670","created_at":"2026-10-04T18:55:29.611Z","trace_propagation_headers":{"sentry-trace":"14446a68ddf1427b8532ebbbfcee3167-e00197410563454a","baggage":"sentry-trace_id=14446a68ddf1427b8532ebbbfcee3167,sentry-release=Unknown"},"correlation_id":"01M444AST3TH8YWE2JKD952NN9","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.614Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5e2d83e28cc2bb9a4c8cf670: done: 0.017074 sec","job_status":"done","queue_duration_s":0.328921,"scheduling_latency_s":0.328941,"redis_calls":8,"redis_duration_s":0.001192,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000935,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000257,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011132,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017074,"completed_at":"2026-10-04T18:55:29.960Z","load_balancing_strategy":"primary","db_duration_s":0.002541,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 253.713297] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.961Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"798971866e0625ede6fb99a8","created_at":"2026-10-04T18:55:29.897Z","trace_propagation_headers":{"sentry-trace":"63638d8da47d47ad999827e21b75e9a7-4153f6f7e7874dd3","baggage":"sentry-trace_id=63638d8da47d47ad999827e21b75e9a7,sentry-release=Unknown"},"correlation_id":"01M444ATE3K12C2WKMCTG9N28S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.899Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-798971866e0625ede6fb99a8: start","job_status":"start","queue_duration_s":0.062146,"scheduling_latency_s":0.062161} gitlab # [ 253.714129] gitaly[916]: time="2026-10-04T18:55:29.606Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444AST3TH8YWE2JKD952NN9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T18:55:59.305" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:29.605" grpc.time_ms=0.66 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 253.714839] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:29.981Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"798971866e0625ede6fb99a8","created_at":"2026-10-04T18:55:29.897Z","trace_propagation_headers":{"sentry-trace":"63638d8da47d47ad999827e21b75e9a7-4153f6f7e7874dd3","baggage":"sentry-trace_id=63638d8da47d47ad999827e21b75e9a7,sentry-release=Unknown"},"correlation_id":"01M444ATE3K12C2WKMCTG9N28S","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:29.899Z","job_size_bytes":3,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-798971866e0625ede6fb99a8: done: 0.020593 sec","job_status":"done","queue_duration_s":0.062146,"scheduling_latency_s":0.062161,"redis_calls":7,"redis_duration_s":0.001007,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000749,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000258,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014807,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020593,"completed_at":"2026-10-04T18:55:29.981Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003064,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.754766] gitaly[916]: time="2026-10-04T18:55:29.909Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444ATE3K12C2WKMCTG9N28S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-04T18:55:59.008" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:29.908" grpc.time_ms=0.821 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 253.797214] gitaly[916]: time="2026-10-04T18:55:29.934Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444ATE3K12C2WKMCTG9N28S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T18:55:59.033" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:29.933" grpc.time_ms=0.532 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 253.797933] gitaly[916]: time="2026-10-04T18:55:30.526Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444AVNA0N001C5CMESHQESN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T18:56:00.425" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:30.525" grpc.time_ms=0.636 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 253.908748] nix-daemon[2811]: accepted connection from pid 3676, user nixbot gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/860c014d5e7635ba9901c9e9bf9260e2b44a4446/statuses gitlab # [ 254.571431] gitaly[916]: time="2026-10-04T18:55:31.660Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444AWRE1DTKNZ8G78YEG8B9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T18:56:01.559" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:31.659" grpc.time_ms=0.869 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 254.603687] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:55:31 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/860c014d5e7635ba9901c9e9bf9260e2b44a4446/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 93 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/860c014d5e7635ba9901c9e9bf9260e2b44a4446/statuses, in 0.13 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/860c014d5e7635ba9901c9e9bf9260e2b44a4446/statuses gitlab # [ 255.705528] gitaly[916]: time="2026-10-04T18:55:32.794Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444AXVVEZFJWWJW0JDXFEWA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T18:56:02.692" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:32.793" grpc.time_ms=0.824 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 255.736292] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:55:32 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/860c014d5e7635ba9901c9e9bf9260e2b44a4446/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 93 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/860c014d5e7635ba9901c9e9bf9260e2b44a4446/statuses, in 0.13 seconds) gitlab # [ 256.205376] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:33.293Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"755cd2e28b7c50cae1faec04","created_at":"2026-10-04T18:55:33.277Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"00ccab9b765374c420b57fe8a232b729","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.291Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-755cd2e28b7c50cae1faec04: start","job_status":"start","queue_duration_s":0.002232,"scheduling_latency_s":0.002325} gitlab # [ 256.284284] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:33.373Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"755cd2e28b7c50cae1faec04","created_at":"2026-10-04T18:55:33.277Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"00ccab9b765374c420b57fe8a232b729","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.291Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-755cd2e28b7c50cae1faec04: done: 0.079499 sec","job_status":"done","queue_duration_s":0.002232,"scheduling_latency_s":0.002325,"redis_calls":11,"redis_duration_s":0.011145,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001729,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005065,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004351,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023314,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000840331,"exclusive_lock_hold_duration_s":0.009073779,"duration_s":0.079499,"completed_at":"2026-10-04T18:55:33.373Z","load_balancing_strategy":"primary","db_duration_s":0.007091,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.357843] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:33.446Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"f41aad4bf2b074c85b94a5de","created_at":"2026-10-04T18:55:33.334Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.445Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-f41aad4bf2b074c85b94a5de: start","job_status":"start","queue_duration_s":0.001591,"scheduling_latency_s":0.001643} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/860c014d5e7635ba9901c9e9bf9260e2b44a4446/statuses gitlab # [ 256.779245] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:33.867Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"f41aad4bf2b074c85b94a5de","created_at":"2026-10-04T18:55:33.334Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.445Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-f41aad4bf2b074c85b94a5de: done: 0.420546 sec","job_status":"done","queue_duration_s":0.001591,"scheduling_latency_s":0.001643,"redis_calls":70,"redis_duration_s":0.18964799999999998,"redis_read_bytes":118,"redis_write_bytes":42530,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.014153,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001592,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.170141,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.003762,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.049,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.049,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.142326,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.420546,"completed_at":"2026-10-04T18:55:33.867Z","load_balancing_strategy":"primary","db_duration_s":0.048493,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.823394] gitaly[916]: time="2026-10-04T18:55:33.912Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444AYYZEHQWJ56T654VFYRQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T18:56:03.810" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:33.911" grpc.time_ms=0.841 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 256.843415] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:55:33 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/860c014d5e7635ba9901c9e9bf9260e2b44a4446/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 77 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/860c014d5e7635ba9901c9e9bf9260e2b44a4446/statuses, in 0.12 seconds) gitlab # [ 256.864795] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:33.954Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"c8d31c76fc308538c0c84b7e","created_at":"2026-10-04T18:55:33.511Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9af2ace4653a900bfc8f9581c243652c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.526Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-c8d31c76fc308538c0c84b7e: start","job_status":"start","queue_duration_s":0.427983,"scheduling_latency_s":0.428015} gitlab # [ 256.997091] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:34.086Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"c8d31c76fc308538c0c84b7e","created_at":"2026-10-04T18:55:33.511Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9af2ace4653a900bfc8f9581c243652c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.526Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-c8d31c76fc308538c0c84b7e: done: 0.1321 sec","job_status":"done","queue_duration_s":0.427983,"scheduling_latency_s":0.428015,"redis_calls":11,"redis_duration_s":0.013665,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00383,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005482,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004353,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033139,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001076674,"exclusive_lock_hold_duration_s":0.059358103,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.1321,"completed_at":"2026-10-04T18:55:34.086Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.057888,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.039945] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:34.128Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"f7fe82c1b92dbc674915641b","created_at":"2026-10-04T18:55:33.538Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fcda0e883926b77f4f3253af62497b5b","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.556Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-f7fe82c1b92dbc674915641b: start","job_status":"start","queue_duration_s":0.572182,"scheduling_latency_s":0.572207} gitlab # [ 257.160828] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:34.249Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"f7fe82c1b92dbc674915641b","created_at":"2026-10-04T18:55:33.538Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fcda0e883926b77f4f3253af62497b5b","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.556Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-f7fe82c1b92dbc674915641b: done: 0.12028 sec","job_status":"done","queue_duration_s":0.572182,"scheduling_latency_s":0.572207,"redis_calls":9,"redis_duration_s":0.03925,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004329,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.032431,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00249,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025678,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.12028,"completed_at":"2026-10-04T18:55:34.249Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024544,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.239185] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:34.256Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"87dff2aad890984ca85f533d","created_at":"2026-10-04T18:55:33.573Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b58153a9bcec0c6f01b58bf13c2520ca","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-04T18:55:33.586Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-87dff2aad890984ca85f533d: start","job_status":"start","queue_duration_s":0.669742,"scheduling_latency_s":0.669784} gitlab # [ 257.251504] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:34.313Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 257.255352] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:34.339Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"87dff2aad890984ca85f533d","created_at":"2026-10-04T18:55:33.573Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b58153a9bcec0c6f01b58bf13c2520ca","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-04T18:55:33.586Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-87dff2aad890984ca85f533d: done: 0.082718 sec","job_status":"done","queue_duration_s":0.669742,"scheduling_latency_s":0.669784,"redis_calls":9,"redis_duration_s":0.009215,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002301,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00643,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000484,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021463,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.082718,"completed_at":"2026-10-04T18:55:34.339Z","load_balancing_strategy":"primary","db_duration_s":0.001294,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.292769] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:34.344Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"ffa4b4fa5458a4a850d522a7","created_at":"2026-10-04T18:55:33.601Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ad0a8ef76461cae9418dbe06bd76f50c","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.614Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-ffa4b4fa5458a4a850d522a7: start","job_status":"start","queue_duration_s":0.729992,"scheduling_latency_s":0.730034} gitlab # [ 257.308988] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:34.398Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"ffa4b4fa5458a4a850d522a7","created_at":"2026-10-04T18:55:33.601Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ad0a8ef76461cae9418dbe06bd76f50c","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.614Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-ffa4b4fa5458a4a850d522a7: done: 0.053664 sec","job_status":"done","queue_duration_s":0.729992,"scheduling_latency_s":0.730034,"redis_calls":11,"redis_duration_s":0.002546,"redis_read_bytes":9,"redis_write_bytes":1174,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000582,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001299,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000245,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00042,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01814,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000315124,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.053664,"completed_at":"2026-10-04T18:55:34.397Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001287,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.350530] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:34.439Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"693bf1ca5a0ef12763057d38","created_at":"2026-10-04T18:55:33.624Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a18631a65634b5b7902132cca296bc9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-04T18:55:33.636Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-693bf1ca5a0ef12763057d38: start","job_status":"start","queue_duration_s":0.803185,"scheduling_latency_s":0.803227} gitlab # [ 257.372826] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:34.462Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 257.385295] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:34.473Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"693bf1ca5a0ef12763057d38","created_at":"2026-10-04T18:55:33.624Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a18631a65634b5b7902132cca296bc9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-04T18:55:33.636Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-693bf1ca5a0ef12763057d38: done: 0.034114 sec","job_status":"done","queue_duration_s":0.803185,"scheduling_latency_s":0.803227,"redis_calls":9,"redis_duration_s":0.008937,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000398,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002967,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005572,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016099,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034114,"completed_at":"2026-10-04T18:55:34.473Z","load_balancing_strategy":"primary","db_duration_s":0.000763,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.424583] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:34.513Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"ba8ee1baa9e9188c9aa1d704","created_at":"2026-10-04T18:55:33.648Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"57a3ff0699c8fd1eed069b0e5d48153e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.661Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-ba8ee1baa9e9188c9aa1d704: start","job_status":"start","queue_duration_s":0.852259,"scheduling_latency_s":0.852302} gitlab # [ 257.472956] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:34.561Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"ba8ee1baa9e9188c9aa1d704","created_at":"2026-10-04T18:55:33.648Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"57a3ff0699c8fd1eed069b0e5d48153e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.661Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-ba8ee1baa9e9188c9aa1d704: done: 0.048255 sec","job_status":"done","queue_duration_s":0.852259,"scheduling_latency_s":0.852302,"redis_calls":11,"redis_duration_s":0.0032300000000000002,"redis_read_bytes":9,"redis_write_bytes":1459,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000987,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001435,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000808,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022931,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.048255,"completed_at":"2026-10-04T18:55:34.561Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001323,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.514848] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:34.604Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"3ecbe866ace3b7eca8de77c5","created_at":"2026-10-04T18:55:33.672Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"29f45a140bd81cca90c28244bc5fe6b4","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.684Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-3ecbe866ace3b7eca8de77c5: start","job_status":"start","queue_duration_s":0.91901,"scheduling_latency_s":0.919063} gitlab # [ 257.769241] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:34.858Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"3ecbe866ace3b7eca8de77c5","created_at":"2026-10-04T18:55:33.672Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"29f45a140bd81cca90c28244bc5fe6b4","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.684Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-3ecbe866ace3b7eca8de77c5: done: 0.254612 sec","job_status":"done","queue_duration_s":0.91901,"scheduling_latency_s":0.919063,"redis_calls":564,"redis_duration_s":0.074474,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000626,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001121,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.072727,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.2024,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.254612,"completed_at":"2026-10-04T18:55:34.858Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000586,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.848805] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:34.938Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"e6d1ffe39e7e2ed9db9f3814","created_at":"2026-10-04T18:55:33.695Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"03e0dda481fb55b4f6eee0a08f2a7762","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.709Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-e6d1ffe39e7e2ed9db9f3814: start","job_status":"start","queue_duration_s":1.228748,"scheduling_latency_s":1.228781} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/860c014d5e7635ba9901c9e9bf9260e2b44a4446/statuses gitlab # [ 257.876171] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:34.965Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 257.888482] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:34.977Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"e6d1ffe39e7e2ed9db9f3814","created_at":"2026-10-04T18:55:33.695Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"03e0dda481fb55b4f6eee0a08f2a7762","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.709Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-e6d1ffe39e7e2ed9db9f3814: done: 0.039454 sec","job_status":"done","queue_duration_s":1.228748,"scheduling_latency_s":1.228781,"redis_calls":9,"redis_duration_s":0.003393,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000522,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001766,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001105,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015001,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.039454,"completed_at":"2026-10-04T18:55:34.977Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00084,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.910262] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:34.979Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"09b195c9cbabe43a9df116a3","created_at":"2026-10-04T18:55:33.732Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"13c03375e838fe761becd352cdee6d3e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-04T18:55:33.745Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-09b195c9cbabe43a9df116a3: start","job_status":"start","queue_duration_s":1.233705,"scheduling_latency_s":1.233743} gitlab # [ 257.949185] gitaly[916]: time="2026-10-04T18:55:35.038Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444B02CDWM6R3ZZHZT4NVVS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T18:56:05.036" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:35.037" grpc.time_ms=0.701 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.984138] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:55:35 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/860c014d5e7635ba9901c9e9bf9260e2b44a4446/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 84 gitlab # [ 257.984522] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.072Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"09b195c9cbabe43a9df116a3","created_at":"2026-10-04T18:55:33.732Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"13c03375e838fe761becd352cdee6d3e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-04T18:55:33.745Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-09b195c9cbabe43a9df116a3: done: 0.093456 sec","job_status":"done","queue_duration_s":1.233705,"scheduling_latency_s":1.233743,"redis_calls":9,"redis_duration_s":0.005552,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000648,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004297,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000607,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.038,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.038,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0292,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.093456,"completed_at":"2026-10-04T18:55:35.072Z","load_balancing_strategy":"primary","db_duration_s":0.047285,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.986260] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.075Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5413f4c93939e5664392e4ea","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5413f4c93939e5664392e4ea: start","job_status":"start","queue_duration_s":1.225491,"scheduling_latency_s":1.225535} gitlab # [ 258.026838] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.106Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5413f4c93939e5664392e4ea","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5413f4c93939e5664392e4ea: done: 0.030849 sec","job_status":"done","queue_duration_s":1.225491,"scheduling_latency_s":1.225535,"redis_calls":15,"redis_duration_s":0.004223,"redis_read_bytes":12,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000574,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001198,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002451,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018589,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.030849,"completed_at":"2026-10-04T18:55:35.106Z","load_balancing_strategy":"primary","db_duration_s":0.000616,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.031850] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.107Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5c80cacf24075c2c801dc332","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5c80cacf24075c2c801dc332: start","job_status":"start","queue_duration_s":1.257866,"scheduling_latency_s":1.257883} gitlab # [ 258.043750] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.119Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5c80cacf24075c2c801dc332","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5c80cacf24075c2c801dc332: done: 0.011314 sec","job_status":"done","queue_duration_s":1.257866,"scheduling_latency_s":1.257883,"redis_calls":13,"redis_duration_s":0.002783,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00082,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001963,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006795,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011314,"completed_at":"2026-10-04T18:55:35.119Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.088132] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.120Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de7d2c35f8adad3bd1d54208","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de7d2c35f8adad3bd1d54208: start","job_status":"start","queue_duration_s":1.270311,"scheduling_latency_s":1.270327} gitlab # [ 258.088979] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.131Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de7d2c35f8adad3bd1d54208","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de7d2c35f8adad3bd1d54208: done: 0.011277 sec","job_status":"done","queue_duration_s":1.270311,"scheduling_latency_s":1.270327,"redis_calls":13,"redis_duration_s":0.002793,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000982,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001811,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006477,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011277,"completed_at":"2026-10-04T18:55:35.131Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.149307] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.132Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a47be88ffbb78e4c43de4d8c","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a47be88ffbb78e4c43de4d8c: start","job_status":"start","queue_duration_s":1.282736,"scheduling_latency_s":1.282771} gitlab # [ 258.150108] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.152Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a47be88ffbb78e4c43de4d8c","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a47be88ffbb78e4c43de4d8c: done: 0.020193 sec","job_status":"done","queue_duration_s":1.282736,"scheduling_latency_s":1.282771,"redis_calls":13,"redis_duration_s":0.003089,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001231,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001858,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014501,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020193,"completed_at":"2026-10-04T18:55:35.152Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.168910] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.155Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1adffd75b0f11e46c49d0f3","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1adffd75b0f11e46c49d0f3: start","job_status":"start","queue_duration_s":1.305409,"scheduling_latency_s":1.305448} gitlab # [ 258.209579] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.173Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1adffd75b0f11e46c49d0f3","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1adffd75b0f11e46c49d0f3: done: 0.018338 sec","job_status":"done","queue_duration_s":1.305409,"scheduling_latency_s":1.305448,"redis_calls":13,"redis_duration_s":0.002307,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000729,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001578,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011704,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018338,"completed_at":"2026-10-04T18:55:35.173Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.213761] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.175Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73ef35d9eadef8759db31a7c","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73ef35d9eadef8759db31a7c: start","job_status":"start","queue_duration_s":1.325231,"scheduling_latency_s":1.325256} gitlab # [ 258.262148] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.191Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73ef35d9eadef8759db31a7c","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73ef35d9eadef8759db31a7c: done: 0.016033 sec","job_status":"done","queue_duration_s":1.325231,"scheduling_latency_s":1.325256,"redis_calls":13,"redis_duration_s":0.002692,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000986,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001706,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008902,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016033,"completed_at":"2026-10-04T18:55:35.191Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.263229] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.192Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11aed30751731d3e54c2246e","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11aed30751731d3e54c2246e: start","job_status":"start","queue_duration_s":1.342943,"scheduling_latency_s":1.342974} gitlab # [ 258.272992] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.203Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11aed30751731d3e54c2246e","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11aed30751731d3e54c2246e: done: 0.010209 sec","job_status":"done","queue_duration_s":1.342943,"scheduling_latency_s":1.342974,"redis_calls":13,"redis_duration_s":0.001617,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00108,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006523,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010209,"completed_at":"2026-10-04T18:55:35.203Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.317817] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.203Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b516855cbe5209b25c661a80","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b516855cbe5209b25c661a80: start","job_status":"start","queue_duration_s":1.353951,"scheduling_latency_s":1.353965} gitlab # [ 258.320276] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.213Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b516855cbe5209b25c661a80","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b516855cbe5209b25c661a80: done: 0.009181 sec","job_status":"done","queue_duration_s":1.353951,"scheduling_latency_s":1.353965,"redis_calls":13,"redis_duration_s":0.0016099999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001085,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005915,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009181,"completed_at":"2026-10-04T18:55:35.213Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.368627] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.214Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3590c4b218ea5dade04ce882","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3590c4b218ea5dade04ce882: start","job_status":"start","queue_duration_s":1.364181,"scheduling_latency_s":1.364195} gitlab # [ 258.369584] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.223Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3590c4b218ea5dade04ce882","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3590c4b218ea5dade04ce882: done: 0.009246 sec","job_status":"done","queue_duration_s":1.364181,"scheduling_latency_s":1.364195,"redis_calls":13,"redis_duration_s":0.001589,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006099,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009246,"completed_at":"2026-10-04T18:55:35.223Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.426704] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.224Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ef6895ac25ce3471a4ec48d","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ef6895ac25ce3471a4ec48d: start","job_status":"start","queue_duration_s":1.374261,"scheduling_latency_s":1.374275} gitlab # [ 258.427973] nix-daemon[2811]: accepted connection from pid 3714, user nixbot gitlab # [ 258.428559] nix-daemon[2811]: accepted connection from pid 3676, user nixbot gitlab # [ 258.429125] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.235Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ef6895ac25ce3471a4ec48d","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ef6895ac25ce3471a4ec48d: done: 0.011455 sec","job_status":"done","queue_duration_s":1.374261,"scheduling_latency_s":1.374275,"redis_calls":13,"redis_duration_s":0.001883,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001313,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007353,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011455,"completed_at":"2026-10-04T18:55:35.235Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.469301] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.236Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4a49d71820e62911e2df355c","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4a49d71820e62911e2df355c: start","job_status":"start","queue_duration_s":1.386818,"scheduling_latency_s":1.386838} gitlab # [ 258.470031] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.246Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4a49d71820e62911e2df355c","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4a49d71820e62911e2df355c: done: 0.009978 sec","job_status":"done","queue_duration_s":1.386818,"scheduling_latency_s":1.386838,"redis_calls":13,"redis_duration_s":0.001791,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001261,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006392,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009978,"completed_at":"2026-10-04T18:55:35.246Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.522885] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.247Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86607857292c6d728c58faed","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86607857292c6d728c58faed: start","job_status":"start","queue_duration_s":1.39761,"scheduling_latency_s":1.397624} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/860c014d5e7635ba9901c9e9bf9260e2b44a4446/statuses, in 0.73 seconds) gitlab # [ 258.523717] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.258Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86607857292c6d728c58faed","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86607857292c6d728c58faed: done: 0.010937 sec","job_status":"done","queue_duration_s":1.39761,"scheduling_latency_s":1.397624,"redis_calls":13,"redis_duration_s":0.0018419999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001318,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006867,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010937,"completed_at":"2026-10-04T18:55:35.258Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.571887] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.259Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c065040cc5267a3a004d61d","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c065040cc5267a3a004d61d: start","job_status":"start","queue_duration_s":1.409492,"scheduling_latency_s":1.409508} gitlab # [ 258.572705] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.269Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c065040cc5267a3a004d61d","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c065040cc5267a3a004d61d: done: 0.009767 sec","job_status":"done","queue_duration_s":1.409492,"scheduling_latency_s":1.409508,"redis_calls":13,"redis_duration_s":0.001679,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001142,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006366,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009767,"completed_at":"2026-10-04T18:55:35.269Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.622166] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.270Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a75c7918edba718bb5856af7","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a75c7918edba718bb5856af7: start","job_status":"start","queue_duration_s":1.42009,"scheduling_latency_s":1.420104} gitlab # [ 258.622915] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.279Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a75c7918edba718bb5856af7","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a75c7918edba718bb5856af7: done: 0.009342 sec","job_status":"done","queue_duration_s":1.42009,"scheduling_latency_s":1.420104,"redis_calls":13,"redis_duration_s":0.001554,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001031,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006272,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009342,"completed_at":"2026-10-04T18:55:35.279Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.669477] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.280Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9684ce80c7bd877f0e9fa493","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9684ce80c7bd877f0e9fa493: start","job_status":"start","queue_duration_s":1.430284,"scheduling_latency_s":1.430297} gitlab # [ 258.670286] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.289Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9684ce80c7bd877f0e9fa493","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9684ce80c7bd877f0e9fa493: done: 0.009643 sec","job_status":"done","queue_duration_s":1.430284,"scheduling_latency_s":1.430297,"redis_calls":13,"redis_duration_s":0.001633,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000597,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001036,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005942,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009643,"completed_at":"2026-10-04T18:55:35.289Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.689072] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.290Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f37cbdb9b7df68e045ecdb37","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f37cbdb9b7df68e045ecdb37: start","job_status":"start","queue_duration_s":1.440771,"scheduling_latency_s":1.440786} gitlab # [ 258.700878] gitaly[916]: time="2026-10-04T18:55:35.724Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444B0R2ND1GRA48X0GWX8X9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T18:56:05.421" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:35.721" grpc.time_ms=2.213 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 258.742156] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.300Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f37cbdb9b7df68e045ecdb37","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f37cbdb9b7df68e045ecdb37: done: 0.009374 sec","job_status":"done","queue_duration_s":1.440771,"scheduling_latency_s":1.440786,"redis_calls":13,"redis_duration_s":0.0015730000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001048,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006259,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009374,"completed_at":"2026-10-04T18:55:35.300Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.787203] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.300Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f297f6172ef7d8c171b1a3cc","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f297f6172ef7d8c171b1a3cc: start","job_status":"start","queue_duration_s":1.450971,"scheduling_latency_s":1.450985} gitlab # [ 258.800805] nixbot[2645]: {"ts": "2026-10-04T18:55:35.818Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/860c014d5e7635ba9901c9e9bf9260e2b44a4446 \"HTTP/1.1 201 Created\""} gitlab # [ 258.800977] gitaly[916]: time="2026-10-04T18:55:35.777Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444B0R2ND1GRA48X0GWX8X9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T18:55:45.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:55:35.771" grpc.time_ms=6.702 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 258.801785] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:55:35 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/860c014d5e7635ba9901c9e9bf9260e2b44a4446 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 134 gitlab # [ 258.802578] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.311Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f297f6172ef7d8c171b1a3cc","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f297f6172ef7d8c171b1a3cc: done: 0.011017 sec","job_status":"done","queue_duration_s":1.450971,"scheduling_latency_s":1.450985,"redis_calls":13,"redis_duration_s":0.002064,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001511,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007061,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011017,"completed_at":"2026-10-04T18:55:35.311Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.849969] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.313Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a345f7d38dea121398855171","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a345f7d38dea121398855171: start","job_status":"start","queue_duration_s":1.463087,"scheduling_latency_s":1.463102} gitlab # [ 258.850833] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.322Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a345f7d38dea121398855171","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a345f7d38dea121398855171: done: 0.009903 sec","job_status":"done","queue_duration_s":1.463087,"scheduling_latency_s":1.463102,"redis_calls":13,"redis_duration_s":0.001648,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001125,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006793,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009903,"completed_at":"2026-10-04T18:55:35.322Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.861255] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.324Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a31e21af8915bfc1d3c4f03","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a31e21af8915bfc1d3c4f03: start","job_status":"start","queue_duration_s":1.474089,"scheduling_latency_s":1.474103} gitlab # [ 258.909617] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.334Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a31e21af8915bfc1d3c4f03","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a31e21af8915bfc1d3c4f03: done: 0.010002 sec","job_status":"done","queue_duration_s":1.474089,"scheduling_latency_s":1.474103,"redis_calls":13,"redis_duration_s":0.0017749999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000606,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001169,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006807,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010002,"completed_at":"2026-10-04T18:55:35.334Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.910825] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.334Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cfcbdb11fe8e6cffc48083b5","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cfcbdb11fe8e6cffc48083b5: start","job_status":"start","queue_duration_s":1.484978,"scheduling_latency_s":1.484991} gitlab # [ 258.917949] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.350Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cfcbdb11fe8e6cffc48083b5","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cfcbdb11fe8e6cffc48083b5: done: 0.015277 sec","job_status":"done","queue_duration_s":1.484978,"scheduling_latency_s":1.484991,"redis_calls":13,"redis_duration_s":0.0016489999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001099,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011386,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015277,"completed_at":"2026-10-04T18:55:35.350Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.953043] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.351Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c39fb725d853e48c9119ca0a","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c39fb725d853e48c9119ca0a: start","job_status":"start","queue_duration_s":1.501169,"scheduling_latency_s":1.501183} gitlab # [ 258.992887] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.363Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c39fb725d853e48c9119ca0a","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c39fb725d853e48c9119ca0a: done: 0.012429 sec","job_status":"done","queue_duration_s":1.501169,"scheduling_latency_s":1.501183,"redis_calls":13,"redis_duration_s":0.002098,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000644,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001454,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008377,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012429,"completed_at":"2026-10-04T18:55:35.363Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.997185] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.364Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"beb1b78f7428d360c96af42b","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-beb1b78f7428d360c96af42b: start","job_status":"start","queue_duration_s":1.514784,"scheduling_latency_s":1.514818} gitlab # [ 259.039806] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.375Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"beb1b78f7428d360c96af42b","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-beb1b78f7428d360c96af42b: done: 0.01132 sec","job_status":"done","queue_duration_s":1.514784,"scheduling_latency_s":1.514818,"redis_calls":13,"redis_duration_s":0.002327,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000889,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001438,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006778,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01132,"completed_at":"2026-10-04T18:55:35.375Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.041422] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.377Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf0e6e7634b2a9bc42a2a982","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf0e6e7634b2a9bc42a2a982: start","job_status":"start","queue_duration_s":1.527466,"scheduling_latency_s":1.52748} gitlab # [ 259.080147] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.387Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf0e6e7634b2a9bc42a2a982","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf0e6e7634b2a9bc42a2a982: done: 0.010072 sec","job_status":"done","queue_duration_s":1.527466,"scheduling_latency_s":1.52748,"redis_calls":13,"redis_duration_s":0.001654,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00113,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006055,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010072,"completed_at":"2026-10-04T18:55:35.387Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.085694] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.388Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d27a6d7f2b5f2833e3b362f","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d27a6d7f2b5f2833e3b362f: start","job_status":"start","queue_duration_s":1.538403,"scheduling_latency_s":1.538416} gitlab # [ 259.130965] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d27a6d7f2b5f2833e3b362f","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d27a6d7f2b5f2833e3b362f: done: 0.009089 sec","job_status":"done","queue_duration_s":1.538403,"scheduling_latency_s":1.538416,"redis_calls":13,"redis_duration_s":0.001622,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001111,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005914,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009089,"completed_at":"2026-10-04T18:55:35.397Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.131873] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.398Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f274f76d1c2672c0b77b9ff5","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f274f76d1c2672c0b77b9ff5: start","job_status":"start","queue_duration_s":1.548357,"scheduling_latency_s":1.548371} gitlab # [ 259.140927] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.407Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f274f76d1c2672c0b77b9ff5","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f274f76d1c2672c0b77b9ff5: done: 0.009163 sec","job_status":"done","queue_duration_s":1.548357,"scheduling_latency_s":1.548371,"redis_calls":13,"redis_duration_s":0.0017360000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000546,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00119,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005796,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009163,"completed_at":"2026-10-04T18:55:35.407Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.185644] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.408Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad4ecfa8c63ef0606c45d2d3","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad4ecfa8c63ef0606c45d2d3: start","job_status":"start","queue_duration_s":1.558369,"scheduling_latency_s":1.558383} gitlab # [ 259.186452] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.417Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad4ecfa8c63ef0606c45d2d3","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad4ecfa8c63ef0606c45d2d3: done: 0.009032 sec","job_status":"done","queue_duration_s":1.558369,"scheduling_latency_s":1.558383,"redis_calls":13,"redis_duration_s":0.001608,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001069,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005677,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009032,"completed_at":"2026-10-04T18:55:35.417Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.238722] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.418Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a479de506ac5c7e642c3957a","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a479de506ac5c7e642c3957a: start","job_status":"start","queue_duration_s":1.568253,"scheduling_latency_s":1.568266} gitlab # [ 259.239700] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.427Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a479de506ac5c7e642c3957a","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a479de506ac5c7e642c3957a: done: 0.009011 sec","job_status":"done","queue_duration_s":1.568253,"scheduling_latency_s":1.568266,"redis_calls":13,"redis_duration_s":0.0015869999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001063,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005647,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009011,"completed_at":"2026-10-04T18:55:35.427Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.289594] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.428Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"35394d3887963a6d5ffb2495","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-35394d3887963a6d5ffb2495: start","job_status":"start","queue_duration_s":1.578248,"scheduling_latency_s":1.578262} gitlab # [ 259.290716] gitaly[916]: time="2026-10-04T18:55:36.311Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444B0R2ND1GRA48X0GWX8X9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:55:36.310" grpc.time_ms=0.503 pid=916 span.kind=server system=grpc gitlab # [ 259.291418] gitaly[916]: time="2026-10-04T18:55:36.334Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444B0R2ND1GRA48X0GWX8X9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:55:36.333" grpc.time_ms=0.438 pid=916 span.kind=server system=grpc gitlab # [ 259.296128] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.437Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"35394d3887963a6d5ffb2495","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-35394d3887963a6d5ffb2495: done: 0.009013 sec","job_status":"done","queue_duration_s":1.578248,"scheduling_latency_s":1.578262,"redis_calls":13,"redis_duration_s":0.001571,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001068,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005669,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009013,"completed_at":"2026-10-04T18:55:35.437Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.338813] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.438Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc5e935234f1f6ef24073a02","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc5e935234f1f6ef24073a02: start","job_status":"start","queue_duration_s":1.588102,"scheduling_latency_s":1.588115} gitlab # [ 259.342700] gitaly[916]: time="2026-10-04T18:55:36.358Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444B0R2ND1GRA48X0GWX8X9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:55:36.358" grpc.time_ms=0.711 pid=916 span.kind=server system=grpc gitlab # [ 259.351654] gitaly[916]: time="2026-10-04T18:55:36.370Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M444B0R2ND1GRA48X0GWX8X9 diskcache=91ef72b0-3e89-4421-89b5-fc00e76c6e3f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-04T18:56:06.059" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:55:36.259" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 259.360805] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.447Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc5e935234f1f6ef24073a02","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc5e935234f1f6ef24073a02: done: 0.009224 sec","job_status":"done","queue_duration_s":1.588102,"scheduling_latency_s":1.588115,"redis_calls":13,"redis_duration_s":0.00178,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001243,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009224,"completed_at":"2026-10-04T18:55:35.447Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.405330] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.448Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"593bad8549402dff1e58269d","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-593bad8549402dff1e58269d: start","job_status":"start","queue_duration_s":1.598168,"scheduling_latency_s":1.598183} gitlab # [ 259.407767] gitaly[916]: time="2026-10-04T18:55:36.370Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=82 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7570 command.oublock=0 command.real_time_ms=71 command.spawn_token_fork_ms=0 command.system_time_ms=55 command.user_time_ms=27 component=gitaly.UnaryServerInterceptor correlation_id=01M444B0R2ND1GRA48X0GWX8X9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-04T18:56:06.059" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:55:36.259" grpc.time_ms=110.666 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 259.419987] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.457Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"593bad8549402dff1e58269d","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-593bad8549402dff1e58269d: done: 0.009073 sec","job_status":"done","queue_duration_s":1.598168,"scheduling_latency_s":1.598183,"redis_calls":13,"redis_duration_s":0.001588,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001084,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00568,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009073,"completed_at":"2026-10-04T18:55:35.457Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.460540] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.458Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c702b6995ae406b010b46b4","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c702b6995ae406b010b46b4: start","job_status":"start","queue_duration_s":1.608151,"scheduling_latency_s":1.608165} gitlab # [ 259.466842] gitaly[916]: time="2026-10-04T18:55:36.415Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444B0R2ND1GRA48X0GWX8X9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-04T18:56:06.114" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:36.414" grpc.time_ms=0.688 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 259.478739] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.467Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c702b6995ae406b010b46b4","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c702b6995ae406b010b46b4: done: 0.009029 sec","job_status":"done","queue_duration_s":1.608151,"scheduling_latency_s":1.608165,"redis_calls":13,"redis_duration_s":0.0016099999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005692,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009029,"completed_at":"2026-10-04T18:55:35.467Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.540805] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.468Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"332768e9caf9b580405f4d61","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-332768e9caf9b580405f4d61: start","job_status":"start","queue_duration_s":1.618045,"scheduling_latency_s":1.618059} gitlab # [ 259.541701] gitaly[916]: time="2026-10-04T18:55:36.435Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444B0R2ND1GRA48X0GWX8X9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T18:56:06.134" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:36.434" grpc.time_ms=0.611 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/860c014d5e7635ba9901c9e9bf9260e2b44a4446/statuses gitlab # [ 259.542579] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.477Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"332768e9caf9b580405f4d61","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-332768e9caf9b580405f4d61: done: 0.009054 sec","job_status":"done","queue_duration_s":1.618045,"scheduling_latency_s":1.618059,"redis_calls":13,"redis_duration_s":0.001662,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000608,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001054,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005706,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009054,"completed_at":"2026-10-04T18:55:35.477Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.611987] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.477Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a163f2a28a8957faed259fb1","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a163f2a28a8957faed259fb1: start","job_status":"start","queue_duration_s":1.627951,"scheduling_latency_s":1.627965} gitlab # [ 259.612843] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.486Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a163f2a28a8957faed259fb1","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a163f2a28a8957faed259fb1: done: 0.008996 sec","job_status":"done","queue_duration_s":1.627951,"scheduling_latency_s":1.627965,"redis_calls":13,"redis_duration_s":0.001565,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001034,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005672,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008996,"completed_at":"2026-10-04T18:55:35.486Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.656042] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.487Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d1a18a7419b9d7d928ad3976","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d1a18a7419b9d7d928ad3976: start","job_status":"start","queue_duration_s":1.637826,"scheduling_latency_s":1.63784} gitlab # [ 259.656742] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.496Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d1a18a7419b9d7d928ad3976","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d1a18a7419b9d7d928ad3976: done: 0.009074 sec","job_status":"done","queue_duration_s":1.637826,"scheduling_latency_s":1.63784,"redis_calls":13,"redis_duration_s":0.001623,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001092,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005674,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009074,"completed_at":"2026-10-04T18:55:35.496Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.703609] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.497Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6794fe30f0586dcaf9ada63a","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6794fe30f0586dcaf9ada63a: start","job_status":"start","queue_duration_s":1.6477,"scheduling_latency_s":1.647713} gitlab # [ 259.704687] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.506Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6794fe30f0586dcaf9ada63a","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6794fe30f0586dcaf9ada63a: done: 0.00897 sec","job_status":"done","queue_duration_s":1.6477,"scheduling_latency_s":1.647713,"redis_calls":13,"redis_duration_s":0.001569,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000499,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00107,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005639,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00897,"completed_at":"2026-10-04T18:55:35.506Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.749740] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.507Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1fd89a4cbd28f1d879c673c5","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1fd89a4cbd28f1d879c673c5: start","job_status":"start","queue_duration_s":1.657527,"scheduling_latency_s":1.65754} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/860c014d5e7635ba9901c9e9bf9260e2b44a4446/statuses, in 0.20 seconds) gitlab # [ 259.750737] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.516Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1fd89a4cbd28f1d879c673c5","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1fd89a4cbd28f1d879c673c5: done: 0.009021 sec","job_status":"done","queue_duration_s":1.657527,"scheduling_latency_s":1.65754,"redis_calls":13,"redis_duration_s":0.001556,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001055,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005679,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009021,"completed_at":"2026-10-04T18:55:35.516Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.762655] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.517Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2118fbd33e96cad530f33f7","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2118fbd33e96cad530f33f7: start","job_status":"start","queue_duration_s":1.667382,"scheduling_latency_s":1.667395} gitlab # [ 259.775519] gitaly[916]: time="2026-10-04T18:55:36.821Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444B1RRP68BM6WK49T9W8HD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T18:56:06.520" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:36.821" grpc.time_ms=0.597 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 259.823367] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.538Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2118fbd33e96cad530f33f7","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2118fbd33e96cad530f33f7: done: 0.020807 sec","job_status":"done","queue_duration_s":1.667382,"scheduling_latency_s":1.667395,"redis_calls":13,"redis_duration_s":0.001992,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000801,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001191,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008367,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020807,"completed_at":"2026-10-04T18:55:35.538Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.824562] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.539Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c10cb44cd589dc6b50a873c5","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c10cb44cd589dc6b50a873c5: start","job_status":"start","queue_duration_s":1.689307,"scheduling_latency_s":1.68933} gitlab # [ 259.869716] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:55:36 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/860c014d5e7635ba9901c9e9bf9260e2b44a4446/statuses HTTP/1.1" 200 677 "" "curl/8.21.0" 147 gitlab # [ 259.870574] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.549Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c10cb44cd589dc6b50a873c5","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c10cb44cd589dc6b50a873c5: done: 0.01025 sec","job_status":"done","queue_duration_s":1.689307,"scheduling_latency_s":1.68933,"redis_calls":13,"redis_duration_s":0.001712,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000565,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001147,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006035,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01025,"completed_at":"2026-10-04T18:55:35.549Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.871816] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.550Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77a4e2237ca3be4f82fd4f99","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77a4e2237ca3be4f82fd4f99: start","job_status":"start","queue_duration_s":1.700398,"scheduling_latency_s":1.700412} gitlab # [ 259.882306] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.570Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77a4e2237ca3be4f82fd4f99","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77a4e2237ca3be4f82fd4f99: done: 0.020343 sec","job_status":"done","queue_duration_s":1.700398,"scheduling_latency_s":1.700412,"redis_calls":13,"redis_duration_s":0.002169,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000636,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001533,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006446,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020343,"completed_at":"2026-10-04T18:55:35.570Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.924638] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.573Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c10a82a4fea631f7d94bcdd","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c10a82a4fea631f7d94bcdd: start","job_status":"start","queue_duration_s":1.723134,"scheduling_latency_s":1.72315} gitlab # [ 259.928192] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.582Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c10a82a4fea631f7d94bcdd","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c10a82a4fea631f7d94bcdd: done: 0.009851 sec","job_status":"done","queue_duration_s":1.723134,"scheduling_latency_s":1.72315,"redis_calls":13,"redis_duration_s":0.001735,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000579,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001156,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00614,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009851,"completed_at":"2026-10-04T18:55:35.582Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 259.969759] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.583Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81fa764e38d38de10aae9c0b","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81fa764e38d38de10aae9c0b: start","job_status":"start","queue_duration_s":1.733896,"scheduling_latency_s":1.733912} gitlab # [ 259.974084] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.594Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81fa764e38d38de10aae9c0b","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81fa764e38d38de10aae9c0b: done: 0.010333 sec","job_status":"done","queue_duration_s":1.733896,"scheduling_latency_s":1.733912,"redis_calls":13,"redis_duration_s":0.0018999999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0007,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0012,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006444,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010333,"completed_at":"2026-10-04T18:55:35.594Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 260.045355] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.595Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d1e97be658f8fe7fd1d169ac","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d1e97be658f8fe7fd1d169ac: start","job_status":"start","queue_duration_s":1.745165,"scheduling_latency_s":1.745179} gitlab # [ 260.046085] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.605Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d1e97be658f8fe7fd1d169ac","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d1e97be658f8fe7fd1d169ac: done: 0.010168 sec","job_status":"done","queue_duration_s":1.745165,"scheduling_latency_s":1.745179,"redis_calls":13,"redis_duration_s":0.001761,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000634,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001127,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006386,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010168,"completed_at":"2026-10-04T18:55:35.605Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 260.056684] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.606Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"830d4e4e3871ecccb03f1233","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-830d4e4e3871ecccb03f1233: start","job_status":"start","queue_duration_s":1.756259,"scheduling_latency_s":1.756276} gitlab # [ 260.097205] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.616Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"830d4e4e3871ecccb03f1233","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-830d4e4e3871ecccb03f1233: done: 0.010397 sec","job_status":"done","queue_duration_s":1.756259,"scheduling_latency_s":1.756276,"redis_calls":13,"redis_duration_s":0.0017430000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001198,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006514,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010397,"completed_at":"2026-10-04T18:55:35.616Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 260.136945] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.617Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0bb56a293d076f9e56cb096d","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0bb56a293d076f9e56cb096d: start","job_status":"start","queue_duration_s":1.76767,"scheduling_latency_s":1.767685} gitlab # [ 260.137696] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.629Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0bb56a293d076f9e56cb096d","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0bb56a293d076f9e56cb096d: done: 0.011666 sec","job_status":"done","queue_duration_s":1.76767,"scheduling_latency_s":1.767685,"redis_calls":13,"redis_duration_s":0.0024289999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000655,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001774,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006799,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011666,"completed_at":"2026-10-04T18:55:35.629Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 260.148566] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.630Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03b4565be837fc22f8fd470e","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03b4565be837fc22f8fd470e: start","job_status":"start","queue_duration_s":1.780234,"scheduling_latency_s":1.780249} gitlab # [ 260.159879] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.639Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03b4565be837fc22f8fd470e","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03b4565be837fc22f8fd470e: done: 0.009482 sec","job_status":"done","queue_duration_s":1.780234,"scheduling_latency_s":1.780249,"redis_calls":13,"redis_duration_s":0.001647,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000558,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001089,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005903,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009482,"completed_at":"2026-10-04T18:55:35.639Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 260.218958] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.640Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df5deaf8248d8211a641beeb","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df5deaf8248d8211a641beeb: start","job_status":"start","queue_duration_s":1.790605,"scheduling_latency_s":1.790619} gitlab # [ 260.219859] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.650Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df5deaf8248d8211a641beeb","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df5deaf8248d8211a641beeb: done: 0.009667 sec","job_status":"done","queue_duration_s":1.790605,"scheduling_latency_s":1.790619,"redis_calls":13,"redis_duration_s":0.001682,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001119,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005854,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009667,"completed_at":"2026-10-04T18:55:35.650Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 260.243280] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.651Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3bea33204d0ca6b8bf514717","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3bea33204d0ca6b8bf514717: start","job_status":"start","queue_duration_s":1.801155,"scheduling_latency_s":1.801169} gitlab # [ 260.283406] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.660Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3bea33204d0ca6b8bf514717","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3bea33204d0ca6b8bf514717: done: 0.009594 sec","job_status":"done","queue_duration_s":1.801155,"scheduling_latency_s":1.801169,"redis_calls":13,"redis_duration_s":0.0016640000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001114,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005835,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009594,"completed_at":"2026-10-04T18:55:35.660Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 260.323293] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.662Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db7082cdb0468e4f3db02b15","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db7082cdb0468e4f3db02b15: start","job_status":"start","queue_duration_s":1.812073,"scheduling_latency_s":1.812088} gitlab # [ 260.324075] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.673Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db7082cdb0468e4f3db02b15","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db7082cdb0468e4f3db02b15: done: 0.011871 sec","job_status":"done","queue_duration_s":1.812073,"scheduling_latency_s":1.812088,"redis_calls":13,"redis_duration_s":0.002157,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000746,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001411,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007137,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011871,"completed_at":"2026-10-04T18:55:35.673Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 260.335773] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.675Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"034ae03d1eecf23834bc33fa","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-034ae03d1eecf23834bc33fa: start","job_status":"start","queue_duration_s":1.82519,"scheduling_latency_s":1.825212} gitlab # [ 260.347216] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.685Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"034ae03d1eecf23834bc33fa","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-034ae03d1eecf23834bc33fa: done: 0.010703 sec","job_status":"done","queue_duration_s":1.82519,"scheduling_latency_s":1.825212,"redis_calls":13,"redis_duration_s":0.001705,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00056,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001145,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006635,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010703,"completed_at":"2026-10-04T18:55:35.685Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 260.397990] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.686Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b6ad706ba337b25165bef0c","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b6ad706ba337b25165bef0c: start","job_status":"start","queue_duration_s":1.836717,"scheduling_latency_s":1.836731} gitlab # [ 260.398770] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.695Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b6ad706ba337b25165bef0c","created_at":"2026-10-04T18:55:33.537Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c1fdab859feb4002","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ebf8501bf3cb25e10a18d1e647da846e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.849Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b6ad706ba337b25165bef0c: done: 0.009134 sec","job_status":"done","queue_duration_s":1.836717,"scheduling_latency_s":1.836731,"redis_calls":13,"redis_duration_s":0.001591,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000495,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005926,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009134,"completed_at":"2026-10-04T18:55:35.695Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 260.438136] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.696Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"36cac8570666d059faa6d49b","created_at":"2026-10-04T18:55:33.837Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fc3afe01e7a4421c074a150b7c899735","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.852Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-36cac8570666d059faa6d49b: start","job_status":"start","queue_duration_s":1.844068,"scheduling_latency_s":1.844082} gitlab # [ 260.480997] gitaly[916]: time="2026-10-04T18:55:37.473Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444B0WDQ9B4DS9BRWE5SA7B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T18:56:07.372" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:37.472" grpc.time_ms=0.573 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 260.481878] nixbot[2645]: {"ts": "2026-10-04T18:55:37.537Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/860c014d5e7635ba9901c9e9bf9260e2b44a4446 \"HTTP/1.1 201 Created\""} gitlab # [ 260.482661] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.719Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"36cac8570666d059faa6d49b","created_at":"2026-10-04T18:55:33.837Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fc3afe01e7a4421c074a150b7c899735","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.852Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-36cac8570666d059faa6d49b: done: 0.022424 sec","job_status":"done","queue_duration_s":1.844068,"scheduling_latency_s":1.844082,"redis_calls":12,"redis_duration_s":0.001871,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000372,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000865,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000634,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015119,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00633656,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.022424,"completed_at":"2026-10-04T18:55:35.719Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002494,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 260.497872] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.720Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"a7307fe9024198255b66a547","created_at":"2026-10-04T18:55:33.865Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab8e3ada0c53b005128b2ca5b13dd311","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.953Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-a7307fe9024198255b66a547: start","job_status":"start","queue_duration_s":1.766597,"scheduling_latency_s":1.766613} gitlab # [ 260.541802] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:55:37 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/860c014d5e7635ba9901c9e9bf9260e2b44a4446 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1714 gitlab # [ 260.542706] gitaly[916]: time="2026-10-04T18:55:37.498Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M444B0WDQ9B4DS9BRWE5SA7B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T18:55:47.062" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:55:37.492" grpc.time_ms=5.89 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 260.543453] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.734Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"a7307fe9024198255b66a547","created_at":"2026-10-04T18:55:33.865Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab8e3ada0c53b005128b2ca5b13dd311","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.953Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-a7307fe9024198255b66a547: done: 0.014509 sec","job_status":"done","queue_duration_s":1.766597,"scheduling_latency_s":1.766613,"redis_calls":11,"redis_duration_s":0.00208,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000295,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000727,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001058,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008127,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000398375,"exclusive_lock_hold_duration_s":0.001511645,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.014509,"completed_at":"2026-10-04T18:55:35.734Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001023,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 260.586743] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.735Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"740fd34ca8f6b0360e07a440","created_at":"2026-10-04T18:55:33.972Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b2849384f4233655cc2b7b0128c3ab34","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.986Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-740fd34ca8f6b0360e07a440: start","job_status":"start","queue_duration_s":1.749264,"scheduling_latency_s":1.749277} gitlab # [ 260.587696] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.742Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 260.588321] gitaly[916]: time="2026-10-04T18:55:37.586Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444B2JD96J412E595JD3X8T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T18:56:07.485" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:37.585" grpc.time_ms=0.674 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 260.594208] nixbot[2645]: {"ts": "2026-10-04T18:55:37.653Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/860c014d5e7635ba9901c9e9bf9260e2b44a4446 \"HTTP/1.1 201 Created\""} gitlab # [ 260.596686] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.746Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"740fd34ca8f6b0360e07a440","created_at":"2026-10-04T18:55:33.972Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b2849384f4233655cc2b7b0128c3ab34","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:33.986Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-740fd34ca8f6b0360e07a440: done: 0.011269 sec","job_status":"done","queue_duration_s":1.749264,"scheduling_latency_s":1.749277,"redis_calls":9,"redis_duration_s":0.001405,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000299,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000813,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000293,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006401,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011269,"completed_at":"2026-10-04T18:55:35.746Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000259,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 260.646389] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.747Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"65f2a73840f2c2309607401d","created_at":"2026-10-04T18:55:33.998Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"362af19f675c34e58208cbaaba9b60b8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:34.013Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-65f2a73840f2c2309607401d: start","job_status":"start","queue_duration_s":1.734326,"scheduling_latency_s":1.734341} gitlab # [ 260.649193] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.754Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 260.652880] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:55:37 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/860c014d5e7635ba9901c9e9bf9260e2b44a4446 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 102 gitlab # [ 260.654944] gitaly[916]: time="2026-10-04T18:55:37.591Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444B0WDQ9B4DS9BRWE5SA7B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-04T18:56:07.289" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:37.590" grpc.time_ms=0.678 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 260.667285] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.758Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"65f2a73840f2c2309607401d","created_at":"2026-10-04T18:55:33.998Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"362af19f675c34e58208cbaaba9b60b8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:34.013Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-65f2a73840f2c2309607401d: done: 0.010981 sec","job_status":"done","queue_duration_s":1.734326,"scheduling_latency_s":1.734341,"redis_calls":9,"redis_duration_s":0.00129,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000309,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000684,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000297,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006409,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010981,"completed_at":"2026-10-04T18:55:35.758Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000254,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 260.720113] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.759Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"1b55ec69e7515a1415e58ef0","created_at":"2026-10-04T18:55:34.056Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"afb3f50753da02b253a9c18b4925959d","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:34.062Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-1b55ec69e7515a1415e58ef0: start","job_status":"start","queue_duration_s":1.697691,"scheduling_latency_s":1.697705} gitlab # [ 260.720764] gitaly[916]: time="2026-10-04T18:55:37.615Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M444B0WDQ9B4DS9BRWE5SA7B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T18:56:07.314" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T18:55:37.614" grpc.time_ms=0.517 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/860c014d5e7635ba9901c9e9bf9260e2b44a4446/statuses gitlab # [ 260.731654] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.783Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"1b55ec69e7515a1415e58ef0","created_at":"2026-10-04T18:55:34.056Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"afb3f50753da02b253a9c18b4925959d","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:34.062Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-1b55ec69e7515a1415e58ef0: done: 0.023127 sec","job_status":"done","queue_duration_s":1.697691,"scheduling_latency_s":1.697705,"redis_calls":10,"redis_duration_s":0.001743,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00027,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001106,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000367,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01223,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000257296,"duration_s":0.023127,"completed_at":"2026-10-04T18:55:35.783Z","load_balancing_strategy":"primary","db_duration_s":0.002879,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 260.783872] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.784Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"79c4f58acc63ee28720858a4","created_at":"2026-10-04T18:55:34.069Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1129986a79237d72c5c14888ef23d200","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:34.073Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-79c4f58acc63ee28720858a4: start","job_status":"start","queue_duration_s":1.710664,"scheduling_latency_s":1.710679} gitlab # [ 260.784775] gitaly[916]: time="2026-10-04T18:55:37.617Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M444B2JD96J412E595JD3X8T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T18:55:47.180" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T18:55:37.611" grpc.time_ms=6.833 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 260.830783] gitaly[916]: time="2026-10-04T18:55:37.732Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444B2JD96J412E595JD3X8T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:55:37.732" grpc.time_ms=0.361 pid=916 span.kind=server system=grpc gitlab # [ 260.831632] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.796Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"79c4f58acc63ee28720858a4","created_at":"2026-10-04T18:55:34.069Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1129986a79237d72c5c14888ef23d200","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:34.073Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-79c4f58acc63ee28720858a4: done: 0.011931 sec","job_status":"done","queue_duration_s":1.710664,"scheduling_latency_s":1.710679,"redis_calls":9,"redis_duration_s":0.001274,"redis_read_bytes":7,"redis_write_bytes":1119,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000258,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00075,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000266,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006948,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011931,"completed_at":"2026-10-04T18:55:35.796Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000246,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 260.881420] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.797Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"52de47f366d50098d4730377","created_at":"2026-10-04T18:55:34.079Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6450cb03ea0fd26c6e9cd62930f204d5","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:34.127Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-52de47f366d50098d4730377: start","job_status":"start","queue_duration_s":1.670137,"scheduling_latency_s":1.670161} gitlab # [ 260.882474] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:55:37 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 45 gitlab # [ 260.884908] gitlab-workhorse[826]: time="2026-10-04T18:55:37Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 260.885651] gitlab-workhorse[826]: gitlab 2001:db8:1::1 - - [2026/10/04:18:55:37 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 48 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/860c014d5e7635ba9901c9e9bf9260e2b44a4446/statuses, in 0.12 seconds) gitlab # [ 260.886243] gitaly[916]: time="2026-10-04T18:55:37.753Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444B2JD96J412E595JD3X8T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:55:37.752" grpc.time_ms=0.655 pid=916 span.kind=server system=grpc (finished: subtest: eval and build post commit statuses, in 8.71 seconds) (finished: run the VM test script, in 311.01 seconds) gitlab # [ 260.886874] gitaly[916]: time="2026-10-04T18:55:37.778Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M444B2JD96J412E595JD3X8T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T18:55:37.777" grpc.time_ms=0.346 pid=916 span.kind=server system=grpc test script finished in 311.06s cleanup kill QemuMachine (pid 46) gitlab # [ 260.888244] sidekiq-cluster[2522]: {"severity":"INFO","time":"2026-10-04T18:55:35.830Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"52de47f366d50098d4730377","created_at":"2026-10-04T18:55:34.079Z","trace_propagation_headers":{"sentry-trace":"b2546fe3e0fa4889ba702169de8653d5-c81ed17c450641bf","baggage":"sentry-trace_id=b2546fe3e0fa4889ba702169de8653d5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6450cb03ea0fd26c6e9cd62930f204d5","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-10-04T18:55:34.127Z","job_size_bytes":2,"pid":2522,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-52de47f366d50098d4730377: done: 0.032743 sec","job_status":"done","queue_duration_s":1.670137,"scheduling_latency_s":1.670161,"redis_calls":28,"redis_duration_s":0.00424,"redis_read_bytes":36,"redis_write_bytes":10593,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000645,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":680,"redis_queues_calls":2,"redis_queues_duration_s":0.000239,"redis_queues_read_bytes":3,"redis_queues_write_bytes":8305,"redis_queues_metadata_calls":16,"redis_queues_metadata_duration_s":0.002362,"redis_queues_metadata_read_bytes":24,"redis_queues_metadata_write_bytes":1162,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000994,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":446,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023191,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count"qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/gxzhl7aaiid7zp3y47jqqiq7zg5mqpwp-python3-3.14.6/bin/python3.14) (finished: cleanup, in 1.71 seconds) additionally exposed symbols: gitlab, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-10-04T18:54:39.174Z', 'push_events': True, 'tag_push_events': True, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] [] [] [] [] [] [] [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '860c014d5e7635ba9901c9e9bf9260e2b44a4446', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-10-04T18:55:29.210094Z', 'started_at': None, 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': {'position': 1, 'blocked_by': []}}] [] [{'id': 5, 'sha': '860c014d5e7635ba9901c9e9bf9260e2b44a4446', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-04T18:55:29.603Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '860c014d5e7635ba9901c9e9bf9260e2b44a4446', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-04T18:55:29.603Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '860c014d5e7635ba9901c9e9bf9260e2b44a4446', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-04T18:55:29.603Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '860c014d5e7635ba9901c9e9bf9260e2b44a4446', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-04T18:55:29.603Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '860c014d5e7635ba9901c9e9bf9260e2b44a4446', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-04T18:55:29.603Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '860c014d5e7635ba9901c9e9bf9260e2b44a4446', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 5s (1 warning)', 'created_at': '2026-10-04T18:55:29.603Z', 'started_at': None, 'finished_at': '2026-10-04T18:55:35.793Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '860c014d5e7635ba9901c9e9bf9260e2b44a4446', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 5s (1 warning)', 'created_at': '2026-10-04T18:55:29.603Z', 'started_at': None, 'finished_at': '2026-10-04T18:55:35.793Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '860c014d5e7635ba9901c9e9bf9260e2b44a4446', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-10-04T18:55:37.518Z', 'started_at': None, 'finished_at': '2026-10-04T18:55:37.632Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]