tribuchet: building on jamie Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.01 seconds) Test will time out and terminate in 3600 seconds run the VM test script start all VMs gitlab: starting vm gitlab # Disk image does not exist, creating the virtualisation disk image... gitlab: QEMU running (pid 46) gitlab # Formatting '/build/vm-state-gitlab/tmp.voSG6808lB', fmt=raw size=1073741824 gitlab # mke2fs 1.47.4 (6-Mar-2025) gitlab # Discarding device blocks: 0/262144 done gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes gitlab # Filesystem UUID: f0779064-e03f-40df-b37f-1a005257c52f (finished: start all VMs, in 75.95 seconds) gitlab # Superblock backups stored on blocks: gitlab # 32768, 98304, 163840, 229376 gitlab # gitlab # Allocating group tables: 0/8 done gitlab: waiting for unit setup-gitlab.service gitlab: waiting for the VM to finish booting gitlab # Writing inode tables: 0/8 done gitlab # Creating journal (8192 blocks): done gitlab # Writing superblocks and filesystem accounting information: 0/8 done gitlab # gitlab # Virtualisation disk image created. gitlab # Creating Nix store image... gitlab # Created Nix store image. gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)... gitlab # gitlab # gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)... gitlab # gitlab # gitlab # Booting from ROM... gitlab # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 6 17:27:09 UTC 2026 gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/c247libqsgzd0myz9v3fq5yffxfhdi2y-nixos-system-gitlab-test/init regInfo=/nix/store/66sdvbcv3ygy3ada28g0as1llil6kbdh-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.000000] BIOS-provided physical RAM map: gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved gitlab # [ 0.000000] NX (Execute Disable) protection: active gitlab # [ 0.000000] APIC: Static calls initialized gitlab # [ 0.000000] SMBIOS 2.8 present. gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 gitlab # [ 0.000000] DMI: Memory slots populated: 1/1 gitlab # [ 0.000000] Hypervisor detected: KVM gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 gitlab # [ 0.000001] kvm-clock: using sched offset of 461764403 cycles gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns gitlab # [ 0.000005] tsc: Detected 2400.012 MHz processor gitlab # [ 0.000809] last_pfn = 0x1c0000 max_arch_pfn = 0x10000000000 gitlab # [ 0.000837] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs gitlab # [ 0.000840] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT gitlab # [ 0.000879] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.002734] found SMP MP-table at [mem 0x000f5470-0x000f547f] gitlab # [ 0.002746] Using GB pages for direct mapping gitlab # [ 0.002855] RAMDISK: [mem 0xbe31c000-0xbffcffff] gitlab # [ 0.002864] ACPI: Early table checksum verification disabled gitlab # [ 0.002867] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) gitlab # [ 0.002871] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002875] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002883] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002885] ACPI: FACS 0x00000000BFFE0000 000040 gitlab # [ 0.002887] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002888] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002890] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002891] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2] gitlab # [ 0.002892] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e] gitlab # [ 0.002893] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] gitlab # [ 0.002893] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482] gitlab # [ 0.002894] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba] gitlab # [ 0.002894] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2] gitlab # [ 0.003132] No NUMA configuration found gitlab # [ 0.003133] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff] gitlab # [ 0.003136] NODE_DATA(0) allocated [mem 0x1bfffa780-0x1bffffcff] gitlab # [ 0.006187] Zone ranges: gitlab # [ 0.006188] DMA [mem 0x0000000000001000-0x0000000000ffffff] gitlab # [ 0.006190] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] gitlab # [ 0.006191] Normal [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.006192] Device empty gitlab # [ 0.006193] Movable zone start for each node gitlab # [ 0.006193] Early memory node ranges gitlab # [ 0.006194] node 0: [mem 0x0000000000001000-0x000000000009efff] gitlab # [ 0.006195] node 0: [mem 0x0000000000100000-0x00000000bffd7fff] gitlab # [ 0.006196] node 0: [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.006197] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff] gitlab # [ 0.006218] On node 0, zone DMA: 1 pages in unavailable ranges gitlab # [ 0.006499] On node 0, zone DMA: 97 pages in unavailable ranges gitlab # [ 0.122724] On node 0, zone Normal: 40 pages in unavailable ranges gitlab # [ 0.123164] ACPI: PM-Timer IO Port: 0x608 gitlab # [ 0.123180] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) gitlab # [ 0.123214] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 gitlab # [ 0.123217] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) gitlab # [ 0.123219] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) gitlab # [ 0.123220] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) gitlab # [ 0.123221] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) gitlab # [ 0.123222] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) gitlab # [ 0.123225] ACPI: Using ACPI (MADT) for SMP configuration information gitlab # [ 0.123226] ACPI: HPET id: 0x8086a201 base: 0xfed00000 gitlab # [ 0.123231] TSC deadline timer available gitlab # [ 0.123235] CPU topo: Max. logical packages: 1 gitlab # [ 0.123236] CPU topo: Max. logical dies: 1 gitlab # [ 0.123236] CPU topo: Max. dies per package: 1 gitlab # [ 0.123240] CPU topo: Max. threads per core: 1 gitlab # [ 0.123241] CPU topo: Num. cores per package: 4 gitlab # [ 0.123241] CPU topo: Num. threads per package: 4 gitlab # [ 0.123241] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs gitlab # [ 0.123266] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() gitlab # [ 0.123285] kvm-guest: KVM setup pv remote TLB flush gitlab # [ 0.123287] kvm-guest: setup PV sched yield gitlab # [ 0.123304] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] gitlab # [ 0.123305] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] gitlab # [ 0.123306] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff] gitlab # [ 0.123308] [mem 0xc0000000-0xfeffbfff] available for PCI devices gitlab # [ 0.123310] Booting paravirtualized kernel on KVM gitlab # [ 0.123314] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns gitlab # [ 0.127780] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 gitlab # [ 0.130108] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288 gitlab # [ 0.130160] kvm-guest: PV spinlocks enabled gitlab # [ 0.130162] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) gitlab # [ 0.130165] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/c247libqsgzd0myz9v3fq5yffxfhdi2y-nixos-system-gitlab-test/init regInfo=/nix/store/66sdvbcv3ygy3ada28g0as1llil6kbdh-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.130264] Unknown kernel command line parameters "regInfo=/nix/store/66sdvbcv3ygy3ada28g0as1llil6kbdh-closure-info/registration", will be passed to user space. gitlab # [ 0.130277] random: crng init done gitlab # [ 0.130278] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes gitlab # [ 0.139721] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) gitlab # [ 0.144938] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) gitlab # [ 0.145019] software IO TLB: area num 4. gitlab # [ 0.225188] Fallback order for Node 0: 0 gitlab # [ 0.225197] Built 1 zonelists, mobility grouping on. Total pages: 1572726 gitlab # [ 0.225199] Policy zone: Normal gitlab # [ 0.227857] mem auto-init: stack:all(zero), heap alloc:on, heap free:off gitlab # [ 0.238270] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 gitlab # [ 0.253233] allocated 12582912 bytes of page_ext gitlab # [ 0.263577] ftrace: allocating 48717 entries in 192 pages gitlab # [ 0.263579] ftrace: allocated 192 pages with 2 groups gitlab # [ 0.264501] Dynamic Preempt: lazy gitlab # [ 0.264717] rcu: Preemptible hierarchical RCU implementation. gitlab # [ 0.264718] rcu: RCU event tracing is enabled. gitlab # [ 0.264719] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. gitlab # [ 0.264721] Trampoline variant of Tasks RCU enabled. gitlab # [ 0.264721] Rude variant of Tasks RCU enabled. gitlab # [ 0.264721] Tracing variant of Tasks RCU enabled. gitlab # [ 0.264722] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. gitlab # [ 0.264723] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 gitlab # [ 0.264745] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.264747] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.264748] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.269191] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16 gitlab # [ 0.269488] rcu: srcu_init: Setting srcu_struct sizes based on contention. gitlab # [ 0.269498] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns gitlab # [ 0.269628] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) gitlab # [ 0.273292] Console: colour VGA+ 80x25 gitlab # [ 0.273295] printk: legacy console [tty0] enabled gitlab # [ 0.304641] printk: legacy console [ttyS0] enabled gitlab # [ 0.412744] ACPI: Core revision 20250807 gitlab # [ 0.413733] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns gitlab # [ 0.415328] APIC: Switch to symmetric I/O mode setup gitlab # [ 0.416381] x2apic enabled gitlab # [ 0.417145] APIC: Switched APIC routing to: physical x2apic gitlab # [ 0.418102] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask() gitlab # [ 0.419403] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself() gitlab # [ 0.420931] kvm-guest: setup PV IPIs gitlab # [ 0.422573] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 gitlab # [ 0.423637] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns gitlab # [ 0.425447] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012) gitlab # [ 0.426535] x86/cpu: User Mode Instruction Prevention (UMIP) activated gitlab # [ 0.427574] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 gitlab # [ 0.429444] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 gitlab # [ 0.430448] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto gitlab # [ 0.432445] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl gitlab # [ 0.433445] Transient Scheduler Attacks: Vulnerable: No microcode gitlab # [ 0.434443] Spectre V2 : Mitigation: Enhanced / Automatic IBRS gitlab # [ 0.435444] Speculative Return Stack Overflow: Mitigation: Safe RET gitlab # [ 0.436444] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization gitlab # [ 0.437453] Spectre V2 : Enabling IBPB for BPF gitlab # [ 0.438445] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier gitlab # [ 0.439445] active return thunk: srso_alias_return_thunk gitlab # [ 0.440367] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' gitlab # [ 0.441444] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' gitlab # [ 0.442444] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' gitlab # [ 0.443444] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' gitlab # [ 0.444444] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' gitlab # [ 0.445444] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' gitlab # [ 0.446444] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' gitlab # [ 0.448443] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 gitlab # [ 0.449444] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 gitlab # [ 0.450444] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 gitlab # [ 0.451444] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 gitlab # [ 0.452444] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 gitlab # [ 0.453444] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format. gitlab # [ 0.484342] Freeing SMP alternatives memory: 44K gitlab # [ 0.485056] pid_max: default: 32768 minimum: 301 gitlab # [ 0.485562] LSM: initializing lsm=capability,landlock,yama,bpf,ima gitlab # [ 0.486566] landlock: Up and running. gitlab # [ 0.487930] Yama: becoming mindful. gitlab # [ 0.488663] LSM support for eBPF active gitlab # [ 0.489702] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.490606] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.494439] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) gitlab # [ 0.495025] Performance Events: Fam17h+ core perfctr, AMD PMU driver. gitlab # [ 0.495449] ... version: 2 gitlab # [ 0.496192] ... bit width: 48 gitlab # [ 0.496446] ... generic counters: 6 gitlab # [ 0.497234] ... generic bitmap: 000000000000003f gitlab # [ 0.498216] ... fixed-purpose counters: 0 gitlab # [ 0.498447] ... fixed-purpose bitmap: 0000000000000000 gitlab # [ 0.499396] ... value mask: 0000ffffffffffff gitlab # [ 0.500167] ... max period: 00007fffffffffff gitlab # [ 0.500446] ... global_ctrl mask: 000000000000003f gitlab # [ 0.501515] signal: max sigframe size: 3376 gitlab # [ 0.502303] rcu: Hierarchical SRCU implementation. gitlab # [ 0.503060] rcu: Max phase no-delay instances is 400. gitlab # [ 0.503645] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level gitlab # [ 0.509023] smp: Bringing up secondary CPUs ... gitlab # [ 0.509872] smpboot: x86: Booting SMP configuration: gitlab # [ 0.510456] .... node #0, CPUs: #1 #2 #3 gitlab # [ 0.513593] smp: Brought up 1 node, 4 CPUs gitlab # [ 0.515171] smpboot: Total of 4 processors activated (19200.09 BogoMIPS) gitlab # [ 0.516009] Memory: 6011392K/6290904K available (17210K kernel code, 2726K rwdata, 13576K rodata, 3640K init, 2992K bss, 260104K reserved, 0K cma-reserved) gitlab # [ 0.518446] devtmpfs: initialized gitlab # [ 0.519577] x86/mm: Memory block size: 128MB gitlab # [ 0.522627] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) gitlab # [ 0.523530] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). gitlab # [ 0.524588] pinctrl core: initialized pinctrl subsystem gitlab # [ 0.526331] PM: RTC time: 10:44:38, date: 2026-10-10 gitlab # [ 0.530344] NET: Registered PF_NETLINK/PF_ROUTE protocol family gitlab # [ 0.532685] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations gitlab # [ 0.533510] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations gitlab # [ 0.535613] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations gitlab # [ 0.536457] audit: initializing netlink subsys (disabled) gitlab # [ 0.537506] audit: type=2000 audit(1791629078.245:1): state=initialized audit_enabled=0 res=1 gitlab # [ 0.537744] thermal_sys: Registered thermal governor 'fair_share' gitlab # [ 0.539447] thermal_sys: Registered thermal governor 'bang_bang' gitlab # [ 0.540437] thermal_sys: Registered thermal governor 'step_wise' gitlab # [ 0.541219] thermal_sys: Registered thermal governor 'user_space' gitlab # [ 0.541446] thermal_sys: Registered thermal governor 'power_allocator' gitlab # [ 0.542460] cpuidle: using governor menu gitlab # [ 0.544697] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 gitlab # [ 0.545725] PCI: Using configuration type 1 for base access gitlab # [ 0.546447] PCI: Using configuration type 1 for extended access gitlab # [ 0.547579] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. gitlab # [ 0.549834] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages gitlab # [ 0.551321] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page gitlab # [ 0.552305] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages gitlab # [ 0.553321] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page gitlab # [ 0.558200] ACPI: Added _OSI(Module Device) gitlab # [ 0.560453] ACPI: Added _OSI(Processor Device) gitlab # [ 0.562454] ACPI: Added _OSI(Processor Aggregator Device) gitlab # [ 0.569349] ACPI: 1 ACPI AML tables successfully acquired and loaded gitlab # [ 0.576232] ACPI: Interpreter enabled gitlab # [ 0.577495] ACPI: PM: (supports S0 S3 S4 S5) gitlab # [ 0.579455] ACPI: Using IOAPIC for interrupt routing gitlab # [ 0.581626] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug gitlab # [ 0.584455] PCI: Using E820 reservations for host bridge windows gitlab # [ 0.587795] ACPI: Enabled 2 GPEs in block 00 to 0F gitlab # [ 0.599482] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) gitlab # [ 0.601504] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] gitlab # [ 0.605351] acpiphp: Slot [3] registered gitlab # [ 0.606430] acpiphp: Slot [4] registered gitlab # [ 0.607425] acpiphp: Slot [5] registered gitlab # [ 0.608168] acpiphp: Slot [6] registered gitlab # [ 0.609473] acpiphp: Slot [7] registered gitlab # [ 0.610184] acpiphp: Slot [8] registered gitlab # [ 0.610480] acpiphp: Slot [9] registered gitlab # [ 0.611194] acpiphp: Slot [10] registered gitlab # [ 0.612474] acpiphp: Slot [11] registered gitlab # [ 0.613201] acpiphp: Slot [12] registered gitlab # [ 0.613480] acpiphp: Slot [13] registered gitlab # [ 0.614194] acpiphp: Slot [14] registered gitlab # [ 0.615466] acpiphp: Slot [15] registered gitlab # [ 0.616187] acpiphp: Slot [16] registered gitlab # [ 0.616474] acpiphp: Slot [17] registered gitlab # [ 0.617194] acpiphp: Slot [18] registered gitlab # [ 0.617465] acpiphp: Slot [19] registered gitlab # [ 0.618183] acpiphp: Slot [20] registered gitlab # [ 0.619475] acpiphp: Slot [21] registered gitlab # [ 0.620200] acpiphp: Slot [22] registered gitlab # [ 0.620466] acpiphp: Slot [23] registered gitlab # [ 0.621198] acpiphp: Slot [24] registered gitlab # [ 0.622466] acpiphp: Slot [25] registered gitlab # [ 0.623182] acpiphp: Slot [26] registered gitlab # [ 0.623466] acpiphp: Slot [27] registered gitlab # [ 0.624191] acpiphp: Slot [28] registered gitlab # [ 0.625467] acpiphp: Slot [29] registered gitlab # [ 0.626182] acpiphp: Slot [30] registered gitlab # [ 0.626465] acpiphp: Slot [31] registered gitlab # [ 0.627198] PCI host bridge to bus 0000:00 gitlab # [ 0.627452] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] gitlab # [ 0.629447] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] gitlab # [ 0.630447] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] gitlab # [ 0.631447] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] gitlab # [ 0.632447] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 0.634447] pci_bus 0000:00: root bus resource [bus 00-ff] gitlab # [ 0.635443] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint gitlab # [ 0.636462] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint gitlab # [ 0.638858] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint gitlab # [ 0.642438] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f] gitlab # [ 0.643160] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk gitlab # [ 0.644447] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk gitlab # [ 0.645447] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk gitlab # [ 0.646447] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk gitlab # [ 0.647969] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint gitlab # [ 0.650475] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f] gitlab # [ 0.653549] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint gitlab # [ 0.654717] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI gitlab # [ 0.656453] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB gitlab # [ 0.657686] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint gitlab # [ 0.672443] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] gitlab # [ 0.673443] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] gitlab # [ 0.675468] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref] gitlab # [ 0.676622] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] gitlab # [ 0.678225] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.683463] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f] gitlab # [ 0.684455] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] gitlab # [ 0.685457] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref] gitlab # [ 0.698437] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] gitlab # [ 0.700697] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.705470] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f] gitlab # [ 0.706404] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] gitlab # [ 0.707219] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref] gitlab # [ 0.710143] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.715455] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf] gitlab # [ 0.716349] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] gitlab # [ 0.717228] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref] gitlab # [ 0.719639] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.724455] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df] gitlab # [ 0.727140] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] gitlab # [ 0.727469] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref] gitlab # [ 0.729681] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.734472] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f] gitlab # [ 0.735433] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] gitlab # [ 0.737473] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref] gitlab # [ 0.739974] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.743484] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff] gitlab # [ 0.744426] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] gitlab # [ 0.746253] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref] gitlab # [ 0.748865] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.752448] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff] gitlab # [ 0.753350] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] gitlab # [ 0.754238] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref] gitlab # [ 0.755418] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref] gitlab # [ 0.758089] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint gitlab # [ 0.761460] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] gitlab # [ 0.762469] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref] gitlab # [ 0.764723] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint gitlab # [ 0.769469] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f] gitlab # [ 0.770377] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] gitlab # [ 0.771245] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref] gitlab # [ 0.777673] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.789626] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f] gitlab # [ 0.792468] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff] gitlab # [ 0.794506] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref] gitlab # [ 0.806818] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 gitlab # [ 0.809933] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 gitlab # [ 0.812789] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 gitlab # [ 0.814776] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 gitlab # [ 0.817652] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 gitlab # [ 0.823464] iommu: Default domain type: Translated gitlab # [ 0.824468] iommu: DMA domain TLB invalidation policy: lazy mode gitlab # [ 0.826849] ACPI: bus type USB registered gitlab # [ 0.827537] usbcore: registered new interface driver usbfs gitlab # [ 0.828462] usbcore: registered new interface driver hub gitlab # [ 0.829376] usbcore: registered new device driver usb gitlab # [ 0.831660] NetLabel: Initializing gitlab # [ 0.832291] NetLabel: domain hash size = 128 gitlab # [ 0.833002] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO gitlab # [ 0.833483] NetLabel: unlabeled traffic allowed by default gitlab # [ 0.834419] PCI: Using ACPI for IRQ routing gitlab # [ 0.835660] pci 0000:00:02.0: vgaarb: setting as boot VGA device gitlab # [ 0.836443] pci 0000:00:02.0: vgaarb: bridge control possible gitlab # [ 0.836443] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none gitlab # [ 0.839450] vgaarb: loaded gitlab # [ 0.840192] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 gitlab # [ 0.840447] hpet0: 3 comparators, 64-bit 100.000000 MHz counter gitlab # [ 0.845535] clocksource: Switched to clocksource kvm-clock gitlab # [ 0.847265] VFS: Disk quotas dquot_6.6.0 gitlab # [ 0.848004] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) gitlab # [ 0.849551] pnp: PnP ACPI init gitlab # [ 0.850770] pnp: PnP ACPI: found 6 devices gitlab # [ 0.859788] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns gitlab # [ 0.861324] clocksource: Switched to clocksource acpi_pm gitlab # [ 0.862317] NET: Registered PF_INET protocol family gitlab # [ 0.864493] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) gitlab # [ 0.882926] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) gitlab # [ 0.884438] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) gitlab # [ 0.885847] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) gitlab # [ 0.890219] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) gitlab # [ 0.891584] TCP: Hash tables configured (established 65536 bind 65536) gitlab # [ 0.892768] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) gitlab # [ 0.894106] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.895554] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.896901] NET: Registered PF_UNIX/PF_LOCAL protocol family gitlab # [ 0.897889] NET: Registered PF_XDP protocol family gitlab # [ 0.898760] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] gitlab # [ 0.899801] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] gitlab # [ 0.900840] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] gitlab # [ 0.901995] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] gitlab # [ 0.903125] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 0.904462] pci 0000:00:01.0: PIIX3: Enabling Passive Release gitlab # [ 0.905454] pci 0000:00:00.0: Limiting direct PCI/PCI transfers gitlab # [ 0.907892] ACPI: \_SB_.LNKD: Enabled at IRQ 11 gitlab # [ 0.910483] PCI: CLS 0 bytes, default 64 gitlab # [ 0.911279] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) gitlab # [ 0.911596] Trying to unpack rootfs image as initramfs... gitlab # [ 0.912344] software IO TLB: mapped [mem 0x00000000ba31c000-0x00000000be31c000] (64MB) gitlab # [ 0.912479] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns gitlab # [ 0.935437] Initialise system trusted keyrings gitlab # [ 0.936727] workingset: timestamp_bits=40 max_order=21 bucket_order=0 gitlab # [ 0.948346] Key type asymmetric registered gitlab # [ 0.949100] Asymmetric key parser 'x509' registered gitlab # [ 0.949989] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) gitlab # [ 0.951682] io scheduler mq-deadline registered gitlab # [ 0.952489] io scheduler kyber registered gitlab # [ 0.957900] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled gitlab # [ 0.959266] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A gitlab # [ 0.961616] Linux agpgart interface v0.103 gitlab # [ 0.962402] ACPI: bus type drm_connector registered gitlab # [ 0.964856] usbcore: registered new interface driver usbserial_generic gitlab # [ 0.965993] usbserial: USB Serial support registered for generic gitlab # [ 0.967010] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. gitlab # [ 0.967010] Please enable it if your BIOS has the CPPC option. gitlab # [ 0.969291] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled gitlab # [ 0.970704] drop_monitor: Initializing network drop monitor service gitlab # [ 0.971884] NET: Registered PF_INET6 protocol family gitlab # [ 0.973653] Segment Routing with IPv6 gitlab # [ 0.974347] In-situ OAM (IOAM) with IPv6 gitlab # [ 0.976573] IPI shorthand broadcast: enabled gitlab # [ 0.981064] sched_clock: Marking stable (833010549, 147945240)->(1007037334, -26081545) gitlab # [ 0.982940] registered taskstats version 1 gitlab # [ 0.984028] Loading compiled-in X.509 certificates gitlab # [ 0.993974] Demotion targets for Node 0: null gitlab # [ 0.995117] Key type .fscrypt registered gitlab # [ 0.995818] Key type fscrypt-provisioning registered gitlab # [ 0.996805] ima: No TPM chip found, activating TPM-bypass! gitlab # [ 0.997747] ima: Allocated hash algorithm: sha1 gitlab # [ 0.998590] ima: No architecture policies found gitlab # [ 0.999749] PM: Magic number: 14:519:729 gitlab # [ 1.001455] RAS: Correctable Errors collector initialized. gitlab # [ 1.005874] clk: Disabling unused clocks gitlab # [ 1.006604] PM: genpd: Disabling unused power domains gitlab # [ 1.195514] Freeing initrd memory: 29392K gitlab # [ 1.202817] Freeing unused decrypted memory: 2028K gitlab # [ 1.209171] Freeing unused kernel image (initmem) memory: 3640K gitlab # [ 1.211794] Write protecting the kernel read-only data: 32768k gitlab # [ 1.216726] Freeing unused kernel image (text/rodata gap) memory: 1220K gitlab # [ 1.220633] Freeing unused kernel image (rodata/data gap) memory: 760K gitlab # [ 1.286367] x86/mm: Checked W+X mappings: passed, no W+X pages found. gitlab # [ 1.287535] Run /init as init process gitlab # [ 1.299796] systemd[1]: Inserted module 'autofs4' gitlab # [ 1.322357] fuse: init (API version 7.45) gitlab # [ 1.330628] ACPI: \_SB_.LNKC: Enabled at IRQ 10 gitlab # [ 1.337133] ACPI: \_SB_.LNKA: Enabled at IRQ 10 gitlab # [ 1.340535] ACPI: \_SB_.LNKB: Enabled at IRQ 11 gitlab # [ 1.378988] systemd[1]: Successfully made /usr/ read-only. gitlab # [ 1.715795] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 1.755286] systemd[1]: Detected virtualization kvm. gitlab # [ 1.757062] systemd[1]: Detected architecture x86-64. gitlab # [ 1.758768] systemd[1]: Running in initrd. gitlab # [ 1.760787] systemd[1]: Initializing machine ID from random generator. gitlab # [ 1.763113] systemd[1]: Hostname set to . gitlab # [ 1.872359] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 1.933866] systemd[1]: Queued start job for default target Initrd Default Target. gitlab # [ 1.942762] systemd[1]: Created slice Slice /system/modprobe. gitlab # [ 1.944020] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 1.945437] systemd[1]: Expecting device /dev/disk/by-label/nix-store... gitlab # [ 1.946583] systemd[1]: Expecting device /dev/disk/by-label/nixos... gitlab # [ 1.947689] systemd[1]: Reached target Path Units. gitlab # [ 1.948571] systemd[1]: Reached target Slice Units. gitlab # [ 1.949453] systemd[1]: Reached target Swaps. gitlab # [ 1.950265] systemd[1]: Reached target Timer Units. gitlab # [ 1.951246] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 1.952460] systemd[1]: Listening on Journal Socket (/dev/log). gitlab # [ 1.953603] systemd[1]: Listening on Journal Sockets. gitlab # [ 1.954698] systemd[1]: Listening on udev Control Socket. gitlab # [ 1.955735] systemd[1]: Listening on udev Kernel Socket. gitlab # [ 1.956694] systemd[1]: Reached target Socket Units. gitlab # [ 1.958849] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 1.961011] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 1.963349] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 1.975516] systemd[1]: Starting Journal Service... gitlab # [ 1.981070] systemd[1]: Starting Load Kernel Modules... gitlab # [ 1.982083] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 1.985096] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 1.989749] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 1.997252] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 2.003317] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 2.009289] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config gitlab # [ 2.020152] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 2.025098] netfs: FS-Cache loaded gitlab # [ 2.026349] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. gitlab # [ 2.033911] systemd-journald[92]: Collecting audit messages is disabled. gitlab # [ 2.034503] 9pnet: Installing 9P2000 support gitlab # [ 2.037046] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev gitlab # [ 2.037140] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 2.041516] systemd[1]: Finished Load Kernel Modules. gitlab # [ 2.042763] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 2.044485] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 2.046883] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 2.048790] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 2.063041] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 2.064778] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 2.066363] systemd[1]: Reached target Local File Systems. gitlab # [ 2.069161] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 2.071787] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 1.924363] systemd-modules-load[93]: Using 4 probe threads gitlab # [ 2.073225] systemd[1]: Started Journal Service. gitlab # [ 1.926574] systemd-modules-load[93]: Inserted module 'virtio_balloon' gitlab # [ 1.928095] systemd-modules-load[93]: Inserted module 'virtio_gpu' gitlab # [ 1.931411] systemd-modules-load[93]: Inserted module 'dm_mod' gitlab # [ 1.932470] systemd[1]: Starting Create System Files and Directories... gitlab # [ 1.944255] systemd[1]: Finished Create System Files and Directories. gitlab # [ 1.948309] systemd-udevd[103]: Using default interface naming scheme 'v261'. gitlab # [ 1.962359] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 1.967489] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 1.968327] systemd[1]: Reached target System Initialization. gitlab # [ 1.969170] systemd[1]: Reached target Basic System. gitlab # [ 2.234978] virtio_blk virtio4: 4/0/0 default/read/poll queues gitlab # [ 2.237727] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) gitlab # [ 2.247828] uhci_hcd 0000:00:01.2: UHCI Host Controller gitlab # [ 2.248840] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1 gitlab # [ 2.249981] uhci_hcd 0000:00:01.2: detected 2 ports gitlab # [ 2.250768] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140 gitlab # [ 2.253014] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 gitlab # [ 2.254275] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 gitlab # [ 2.255548] usb usb1: Product: UHCI Host Controller gitlab # [ 2.256410] usb usb1: Manufacturer: Linux 6.18.43 uhci_hcd gitlab # [ 2.257048] SCSI subsystem initialized gitlab # [ 2.257384] usb usb1: SerialNumber: 0000:00:01.2 gitlab # [ 2.259078] virtio_blk virtio5: 4/0/0 default/read/poll queues gitlab # [ 2.259408] hub 1-0:1.0: USB hub found gitlab # [ 2.260724] hub 1-0:1.0: 2 ports detected gitlab # [ 2.263372] virtio_blk virtio5: [vdb] 11953056 512-byte logical blocks (6.12 GB/5.70 GiB) gitlab # [ 2.305454] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 gitlab # [ 2.310135] serio: i8042 KBD port at 0x60,0x64 irq 1 gitlab # [ 2.310884] serio: i8042 AUX port at 0x60,0x64 irq 12 gitlab # [ 2.318694] scsi host0: ata_piix gitlab # [ 2.320453] scsi host1: ata_piix gitlab # [ 2.321098] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0 gitlab # [ 2.322538] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0 gitlab # [ 2.185186] systemd[1]: Starting Virtual Console Setup... gitlab # [ 2.190666] (udev-worker)[126]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 2.195369] (udev-worker)[126]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.197629] (udev-worker)[123]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.205788] systemd[1]: Found device /dev/disk/by-label/nixos. gitlab # [ 2.207714] systemd[1]: Found device /dev/disk/by-label/nix-store. gitlab # [ 2.208665] systemd[1]: Reached target Initrd Root Device. gitlab # [ 2.209519] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... gitlab # [ 2.210756] systemd-vconsole-setup[150]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 2.212324] systemd[1]: Finished Virtual Console Setup. gitlab # [ 2.243085] systemd-fsck[157]: nixos: clean, 12/65536 files, 13019/262144 blocks gitlab # [ 2.247044] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. gitlab # [ 2.482396] ata2: found unknown device (class 0) gitlab # [ 2.483043] usb 1-1: new full-speed USB device number 2 using uhci_hcd gitlab # [ 2.484799] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 gitlab # [ 2.490522] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 gitlab # [ 2.521152] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 gitlab # [ 2.550300] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray gitlab # [ 2.577685] cdrom: Uniform CD-ROM driver Revision: 3.20 gitlab # [ 2.651766] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 gitlab # [ 2.654757] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 gitlab # [ 2.657501] usb 1-1: Product: QEMU USB Tablet gitlab # [ 2.659201] usb 1-1: Manufacturer: QEMU gitlab # [ 2.660684] usb 1-1: SerialNumber: 28754-0000:00:01.2-1 gitlab # [ 2.691412] hid: raw HID events driver (C) Jiri Kosina gitlab # [ 2.704249] usbcore: registered new interface driver usbhid gitlab # [ 2.705065] usbhid: USB HID core driver gitlab # [ 2.709408] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 gitlab # [ 2.712084] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0 gitlab # [ 2.842521] systemd[1]: Mounting /sysroot... gitlab # [ 3.126484] EXT4-fs (vda): mounted filesystem f0779064-e03f-40df-b37f-1a005257c52f r/w with ordered data mode. Quota mode: none. gitlab # [ 2.984908] systemd[1]: Mounted /sysroot. gitlab # [ 2.986614] systemd[1]: Reached target Initrd Root File System. gitlab # [ 2.988594] systemd[1]: Mounting /sysroot/nix/.ro-store... gitlab # [ 2.990503] systemd[1]: Mounting /sysroot/nix/.rw-store... gitlab # [ 2.992423] systemd[1]: Mounting /sysroot/run... gitlab # [ 2.998296] systemd[1]: Mounting /sysroot/tmp/shared... gitlab # [ 3.003746] systemd[1]: Mounting /sysroot/tmp/xchg... gitlab # [ 3.010184] systemd[1]: Starting Mountpoints Configured in the Real Root... gitlab # [ 3.159564] erofs (device vdb): mounted with root inode @ nid 128. gitlab # [ 3.017321] systemd[1]: Mounted /sysroot/nix/.ro-store. gitlab # [ 3.018301] systemd[1]: Mounted /sysroot/nix/.rw-store. gitlab # [ 3.019155] systemd[1]: Mounted /sysroot/run. gitlab # [ 3.023070] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.172663] 9p: Installing v9fs 9p2000 file system support gitlab # [ 3.026104] systemd-sysroot-fstab-check[178]: /sysroot should be mounted in the initrd, will request daemon-reload. gitlab # [ 3.028628] systemd[1]: Reload requested from client PID 178 ('systemd-sysroot') (unit initrd-parse-etc.service)... gitlab # [ 3.030910] systemd[1]: Reloading... gitlab # [ 3.091641] systemd[1]: Reloading finished in 61 ms. gitlab # [ 3.115710] systemd-sysroot-fstab-check[178]: Requesting initrd-fs.target/start/replace... gitlab # [ 3.118791] systemd[1]: Mounted /sysroot/tmp/shared. gitlab # [ 3.120735] systemd[1]: Mounted /sysroot/tmp/xchg. gitlab # [ 3.122736] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.125344] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.127390] systemd-sysroot-fstab-check[178]: Requesting swap.target/start/replace... gitlab # [ 3.130071] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.132211] systemd[1]: initrd-parse-etc.service: Deactivated successfully. gitlab # [ 3.134597] systemd[1]: Finished Mountpoints Configured in the Real Root. gitlab # [ 3.136926] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. gitlab # [ 3.145774] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.148703] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.842991] systemd[1]: Mounting /sysroot/nix/store... gitlab # [ 3.898160] systemd[1]: Mounted /sysroot/nix/store. gitlab # [ 3.901062] systemd[1]: Reached target Initrd File Systems. gitlab # [ 3.904880] systemd[1]: Starting Find NixOS closure... gitlab # [ 3.908344] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... gitlab # [ 3.934710] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. gitlab # [ 3.938997] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. gitlab # [ 3.953202] systemd[1]: Finished Find NixOS closure. gitlab # [ 3.954990] systemd[1]: Reached target Initrd Default Target. gitlab # [ 3.957063] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... gitlab # [ 3.993407] systemd[1]: Stopped target Initrd Default Target. gitlab # [ 3.995412] systemd[1]: Stopped target Basic System. gitlab # [ 3.997175] systemd[1]: Stopped target Initrd Root Device. gitlab # [ 3.998983] systemd[1]: Stopped target Path Units. gitlab # [ 4.000722] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. gitlab # [ 4.003224] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. gitlab # [ 4.005700] systemd[1]: Stopped target Slice Units. gitlab # [ 4.007386] systemd[1]: Stopped target Socket Units. gitlab # [ 4.009450] systemd[1]: Stopped target System Initialization. gitlab # [ 4.013071] systemd[1]: Stopped target Swaps. gitlab # [ 4.016895] systemd[1]: Stopped target Timer Units. gitlab # [ 4.020384] systemd[1]: dbus.socket: Deactivated successfully. gitlab # [ 4.024422] systemd[1]: Closed D-Bus System Message Bus Socket. gitlab # [ 4.026673] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. gitlab # [ 4.029189] systemd[1]: Stopped Find NixOS closure. gitlab # [ 4.030990] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 4.034588] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 4.036539] systemd[1]: systemd-sysctl.service: Deactivated successfully. gitlab # [ 4.038445] systemd[1]: Stopped Apply Kernel Variables. gitlab # [ 4.039834] systemd[1]: systemd-modules-load.service: Deactivated successfully. gitlab # [ 4.041687] systemd[1]: Stopped Load Kernel Modules. gitlab # [ 4.043097] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. gitlab # [ 4.045107] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. gitlab # [ 4.047112] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. gitlab # [ 4.048989] systemd[1]: Stopped Create System Files and Directories. gitlab # [ 4.050706] systemd[1]: Stopped target Local File Systems. gitlab # [ 4.052190] systemd[1]: Stopped target Preparation for Local File Systems. gitlab # [ 4.053937] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. gitlab # [ 4.055229] systemd[1]: Stopped Coldplug All udev Devices. gitlab # [ 4.056280] systemd[1]: Stopping Rule-based Manager for Device Events and Files... gitlab # [ 4.057560] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 4.058849] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 4.059870] systemd[1]: initrd-cleanup.service: Deactivated successfully. gitlab # [ 4.061235] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. gitlab # [ 4.062602] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 4.063814] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 4.064625] systemd[1]: systemd-udevd.service: Deactivated successfully. gitlab # [ 4.065507] systemd[1]: Stopped Rule-based Manager for Device Events and Files. gitlab # [ 4.066563] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. gitlab # [ 4.067614] systemd[1]: Closed udev Control Socket. gitlab # [ 4.068326] systemd[1]: Starting Cleanup udev Database... gitlab # [ 4.069117] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. gitlab # [ 4.070212] systemd[1]: Stopped Create Static Device Nodes in /dev. gitlab # [ 4.071086] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. gitlab # [ 4.072233] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. gitlab # [ 4.073224] systemd[1]: kmod-static-nodes.service: Deactivated successfully. gitlab # [ 4.074282] systemd[1]: Stopped Create List of Static Device Nodes. gitlab # [ 4.079160] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. gitlab # [ 4.080240] systemd[1]: Finished Cleanup udev Database. gitlab # [ 4.080992] systemd[1]: Reached target Switch Root. gitlab # [ 4.081781] systemd[1]: Starting NixOS Activation... gitlab # [ 4.199392] initrd-nixos-activation-start[228]: booting system configuration /nix/store/c247libqsgzd0myz9v3fq5yffxfhdi2y-nixos-system-gitlab-test gitlab # [ 4.227776] initrd-nixos-activation-start[228]: running activation script... gitlab # [ 4.435511] initrd-nixos-activation-start[251]: setting up /etc... gitlab # [ 4.546219] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. gitlab # [ 4.547434] systemd[1]: Finished NixOS Activation. gitlab # [ 4.549857] systemd[1]: Starting Switch Root... gitlab # [ 4.580656] systemd[1]: Switching root. gitlab # [ 4.792806] systemd-journald[92]: Received SIGTERM from PID 1 (systemd). gitlab # [ 4.916263] NET: Registered PF_VSOCK protocol family gitlab # [ 5.293228] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 5.303274] systemd[1]: Detected virtualization kvm. gitlab # [ 5.305105] systemd[1]: Detected architecture x86-64. gitlab # [ 5.307120] systemd[1]: Detected first boot. gitlab # [ 5.309771] systemd[1]: Initializing machine ID from random generator. gitlab # [ 5.452483] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 5.552588] systemd[1]: Applying preset policy. gitlab # [ 5.735348] systemd[1]: Populated /etc with preset unit settings. gitlab # [ 5.850321] systemd[1]: initrd-switch-root.service: Deactivated successfully. gitlab # [ 5.851698] systemd[1]: Stopped initrd-switch-root.service. gitlab # [ 5.853891] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. gitlab # [ 5.856108] systemd[1]: Created slice Slice /system/getty. gitlab # [ 5.857714] systemd[1]: Created slice GitLab DevOps Platform Slice. gitlab # [ 5.859261] systemd[1]: Created slice User and Session Slice. gitlab # [ 5.860312] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 5.861621] systemd[1]: Started Forward Password Requests to Wall Directory Watch. gitlab # [ 5.862748] systemd[1]: Expecting device /dev/hvc0... gitlab # [ 5.863497] systemd[1]: Expecting device /dev/ttyS0... gitlab # [ 5.864354] systemd[1]: Reached target Local Encrypted Volumes. gitlab # [ 5.865240] systemd[1]: Stopped target initrd-fs.target. gitlab # [ 5.866055] systemd[1]: Stopped target initrd-root-fs.target. gitlab # [ 5.866847] systemd[1]: Stopped target initrd-switch-root.target. gitlab # [ 5.867713] systemd[1]: Reached target Virtual Machines and Containers. gitlab # [ 5.868680] systemd[1]: Reached target Path Units. gitlab # [ 5.869423] systemd[1]: Reached target Remote File Systems. gitlab # [ 5.870380] systemd[1]: Reached target Slice Units. gitlab # [ 5.871168] systemd[1]: Reached target Swaps. gitlab # [ 5.872618] systemd[1]: Listening on Query the User Interactively for a Password. gitlab # [ 5.874763] systemd[1]: Listening on Process Core Dump Socket. gitlab # [ 5.876195] systemd[1]: Listening on Credential Encryption/Decryption. gitlab # [ 5.877818] systemd[1]: Listening on Factory Reset Management. gitlab # [ 5.878811] systemd[1]: Listening on Hostname Service Socket. gitlab # [ 5.881296] systemd[1]: Starting Journal Log Access Socket... gitlab # [ 5.882732] systemd[1]: Listening on Journal Audit Socket. gitlab # [ 5.884747] systemd[1]: Listening on Console Output Muting Service Socket. gitlab # [ 5.885987] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. gitlab # [ 5.887142] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.888531] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 5.891369] systemd[1]: Listening on Disk Repartitioning Service Socket. gitlab # [ 5.892501] systemd[1]: Listening on udev Control Socket. gitlab # [ 5.893417] systemd[1]: Listening on udev Varlink Socket. gitlab # [ 5.896697] systemd[1]: Mounting Huge Pages File System... gitlab # [ 5.898688] systemd[1]: Mounting POSIX Message Queue File System... gitlab # [ 5.900688] systemd[1]: Mounting Kernel Debug File System... gitlab # [ 5.903928] systemd[1]: Mounting Kernel Trace File System... gitlab # [ 5.916161] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 5.917308] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 5.921612] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 5.956749] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm gitlab # [ 5.960792] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 5.964851] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 5.970508] systemd[1]: Mounting FUSE Control File System... gitlab # [ 5.972313] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 5.980026] systemd[1]: Starting Journal Service... gitlab # [ 5.983567] systemd[1]: Starting Load Kernel Modules... gitlab # [ 5.987695] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... gitlab # [ 5.996029] systemd[1]: Starting Remount Root and Kernel File Systems... gitlab # [ 5.997069] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.001019] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 6.003767] systemd[1]: Listening on Journal Log Access Socket. gitlab # [ 6.007403] systemd[1]: Mounted Huge Pages File System. gitlab # [ 6.008508] systemd[1]: Mounted POSIX Message Queue File System. gitlab # [ 6.008573] systemd-journald[321]: Collecting audit messages is enabled. gitlab # [ 6.008876] systemd[1]: Mounted Kernel Debug File System. gitlab # [ 6.011305] systemd[1]: Mounted Kernel Trace File System. gitlab # [ 6.013473] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 6.014838] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 6.016034] EXT4-fs (vda): re-mounted f0779064-e03f-40df-b37f-1a005257c52f. gitlab # [ 6.016243] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 6.019267] systemd[1]: Mounted FUSE Control File System. gitlab # [ 6.020656] systemd[1]: Finished Remount Root and Kernel File Systems. gitlab # [ 6.022230] systemd[1]: Listening on Disk Image Download Service Socket. gitlab # [ 5.879544] systemd[1]: Queued start job for default target Multi-User System. gitlab # [ 5.881323] systemd[1]: Unnecessary job was removed for /dev/vdb. gitlab # [ 5.883381] systemd[1]: systemd-journald.service: Deactivated successfully. gitlab # [ 6.031129] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 5.884729] systemd-modules-load[322]: Using 3 probe threads gitlab # [ 5.886140] systemd-modules-load[322]: Insert[ 6.033863] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # ed module 'tls'[ 6.034146] loop: module loaded gitlab # gitlab # [ 5.889350] systemd-modules-load[322]: Inserted module 'loop' gitlab # [ 5.898891] systemd-oomd[323]: No swap; memory pressure usage will be degraded gitlab # [ 6.052913] systemd[1]: Starting Load/Save OS Random Seed... gitlab # [ 6.055508] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 6.056643] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.058795] systemd[1]: Started Journal Service. gitlab # [ 5.912997] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. gitlab # [ 5.914675] systemd[1]: Finished Load Kernel Modules. gitlab # [ 5.915476] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 5.917773] systemd[1]: Finished Load/Save OS Random Seed. gitlab # [ 5.920825] systemd[1]: Reached target First Boot Complete. gitlab # [ 5.922434] systemd[1]: Starting Firewall... gitlab # [ 5.924090] systemd[1]: Starting Flush Journal to Persistent Storage... gitlab # [ 5.931133] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 5.934250] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 6.084139] systemd-journald[321]: Received client request to flush runtime journal. gitlab # [ 5.966686] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 5.968430] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 5.970216] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 5.971274] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 5.972254] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 5.973316] systemd[1]: Finished Flush Journal to Persistent Storage. gitlab # [ 5.980978] systemd-udevd[354]: Using default interface naming scheme 'v261'. gitlab # [ 6.003954] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 6.011557] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 6.075209] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 6.125448] (udev-worker)[433]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.138124] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. gitlab # [ 6.146582] (udev-worker)[421]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 6.148987] (udev-worker)[421]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.162166] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. gitlab # [ 6.316380] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 gitlab # [ 6.344384] ACPI: button: Power Button [PWRF] gitlab # [ 6.353354] rtc_cmos 00:05: RTC can wake from S4 gitlab # [ 6.353444] parport_pc 00:03: reported by Plug and Play ACPI gitlab # [ 6.355096] rtc_cmos 00:05: registered as rtc0 gitlab # [ 6.355167] parport0: PC-style at 0x378 gitlab # [ 6.355433] rtc_cmos 00:05: setting system clock to 2026-10-10T10:44:43 UTC (1791629083) gitlab # [ 6.355846] , irq 7 [PCSPP(,...)] gitlab # [ 6.357952] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs gitlab # [ 6.360535] Floppy drive(s): fd0 is 2.88M AMI BIOS gitlab # [ 6.362253] systemd-journald[321]: Time jumped backwards, rotating. gitlab # [ 6.372328] FDC 0 is a S82078B gitlab # [ 6.374594] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console gitlab # [ 6.233268] systemd[1]: Finished Firewall. gitlab # [ 6.237655] systemd[1]: Condition check resulted in Virtio network device being skipped. gitlab # [ 6.239494] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 6.241819] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 6.245479] s[ 6.393444] Console: switching to colour dummy device 80x25 gitlab # ystemd[1]: Platform Persistent Storage Archival [ 6.394432] [drm] Found bochs VGA, ID 0xb0c5. gitlab # skipped, unmet c[ 6.394433] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. gitlab # ondition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 6.249166] s[ 6.397256] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic gitlab # ystemd[1]: Early[ 6.398102] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0 gitlab # TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.252390] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.411591] Console: switching to colour frame buffer device 160x50 gitlab # [ 6.412899] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device gitlab # [ 6.416157] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 gitlab # [ 6.417313] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD gitlab # [ 6.417584] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input4 gitlab # [ 6.445474] mousedev: PS/2 mouse device common for all mice gitlab # [ 6.342470] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.510858] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6 gitlab # [ 6.512054] ppdev: user-space parallel port driver gitlab # [ 6.368860] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 6.371097] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 6.375157] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.392308] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 6.394452] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 6.549327] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 gitlab # [ 6.401807] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.575010] kvm_amd: TSC scaling supported gitlab # [ 6.575522] kvm_amd: Nested Virtualization enabled gitlab # [ 6.575525] kvm_amd: Nested Paging enabled gitlab # [ 6.575529] kvm_amd: LBR virtualization supported gitlab # [ 6.575535] kvm_amd: Virtual VMLOAD VMSAVE supported gitlab # [ 6.577586] kvm_amd: Virtual GIF supported gitlab # [ 6.579000] kvm_amd: Virtual NMI enabled gitlab # [ 6.611303] EDAC MC: Ver: 3.0.0 gitlab # [ 6.478491] systemd-vconsole-setup[493]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 6.480974] systemd[1]: Finished Virtual Console Setup. gitlab # [ 6.707512] systemd[1]: Mounting /run/wrappers... gitlab # [ 6.730318] systemd[1]: Mounted /run/wrappers. gitlab # [ 6.732889] systemd[1]: Reached target Local File Systems. gitlab # [ 6.736216] systemd[1]: Listening on Boot Loader Control Service Socket. gitlab # [ 6.740736] systemd[1]: Starting register-nix-paths.service... gitlab # [ 6.744872] systemd[1]: Starting Create SUID/SGID Wrappers... gitlab # [ 6.746887] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 6.755438] systemd[1]: Starting Save Transient machine-id to Disk... gitlab # [ 6.761143] systemd[1]: Starting Create System Files and Directories... gitlab # [ 6.786198] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. gitlab # [ 6.790678] systemd[1]: Finished Save Transient machine-id to Disk. gitlab # [ 6.801256] systemd[1]: Finished Create System Files and Directories. gitlab # [ 6.806692] systemd[1]: Starting Rebuild Journal Catalog... gitlab # [ 6.808513] systemd[1]: Starting Record System Boot/Shutdown in UTMP... gitlab # [ 6.829395] systemd[1]: Finished Record System Boot/Shutdown in UTMP. gitlab # [ 6.845736] systemd[1]: Finished Rebuild Journal Catalog. gitlab # [ 6.847907] systemd[1]: Starting Update is Completed... gitlab # [ 6.863163] systemd[1]: Finished Update is Completed. gitlab # [ 6.908605] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. gitlab # [ 6.910355] systemd[1]: Finished Create SUID/SGID Wrappers. gitlab # [ 7.129991] systemd[1]: Finished register-nix-paths.service. gitlab # [ 7.132396] systemd[1]: Reached target System Initialization. gitlab # [ 7.134481] systemd[1]: Started Discard unused filesystem blocks once a week. gitlab # [ 7.136853] systemd[1]: Started logrotate.timer. gitlab # [ 7.138512] systemd[1]: Started Daily Cleanup of Temporary Directories. gitlab # [ 7.140727] systemd[1]: Reached target Timer Units. gitlab # [ 7.142483] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 7.144593] systemd[1]: Listening on Nix Daemon Socket. gitlab # [ 7.151177] systemd[1]: Starting nixbot CI service socket... gitlab # [ 7.153262] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. gitlab # [ 7.156095] systemd[1]: Listening on nixbot CI service socket. gitlab # [ 7.158108] systemd[1]: Reached target Socket Units. gitlab # [ 7.159805] systemd[1]: Starting D-Bus System Message Bus... gitlab # [ 7.182779] dbus-broker-launch[578]: Looking up NSS user entry for 'systemd-timesync'... gitlab # [ 7.186055] dbus-broker-launch[578]: NSS returned no entry for 'systemd-timesync' gitlab # [ 7.188807] dbus-broker-launch[578]: Invalid user-name in /nix/store/37yal9w16hfdxjh64sww8gn81f4hma6i-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" gitlab # [ 7.193713] systemd[1]: Started D-Bus System Message Bus. gitlab # [ 7.197095] dbus-broker-launch[578]: Ready gitlab # [ 7.199599] systemd[1]: Reached target Basic System. gitlab # [ 7.202554] systemd[1]: Started backdoor.service. gitlab # [ 7.205243] systemd[1]: Starting gitlab-config.service... gitlab # [ 7.207887] systemd[1]: Starting Import lastlog data into lastlog2 database... gitlab # [ 7.218350] systemd[1]: Starting Logrotate configuration check... gitlab # [ 7.222751] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.227234] systemd[1]: Starting Post-Boot Actions... gitlab # [ 7.233873] systemd[1]: Started Reset console on configuration changes. gitlab # [ 7.240218] systemd[1]: Starting resolvconf update... gitlab # [ 7.242745] logrotate[583]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. gitlab # connecting to host... gitlab # [ 7.246739] logrotate[583]: reading config file /nix/store/y3500wq742i80l631mh8nc8rchpsfb9q-logrotate.conf gitlab # [ 7.250252] logrotate[583]: note: 'monthly' overrides previously specified 'weekly' gitlab # [ 7.250869] logrotate[583]: Reading state from file: /var/lib/logrotate.status gitlab # [ 7.251771] logrotate[583]: state file /var/lib/logrotate.status does not exist gitlab # [ 7.254667] logrotate[583]: Allocating hash table for state file, size 64 entries gitlab # [ 7.255691] logrotate[583]: Handling 4 logs gitlab # [ 7.255764] logrotate[583]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.255832] logrotate[583]: considering log /var/log/btmp gitlab # [ 7.255945] logrotate[583]: Creating new state gitlab # [ 7.256032] logrotate[583]: Now: 2026-10-10 10:44 gitlab # [ 7.256100] logrotate[583]: Last rotated at 2026-10-10 10:00 gitlab # [ 7.256190] logrotate[583]: log does not need rotating (log has already been rotated) gitlab # [ 7.256262] logrotate[583]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.256371] logrotate[583]: considering log /var/log/wtmp gitlab # [ 7.256443] logrotate[583]: Creating new state gitlab # [ 7.256507] logrotate[583]: Now: 2026-10-10 10:44 gitlab # [ 7.256579] logrotate[583]: Last rotated at 2026-10-10 10:00 gitlab # [ 7.256642] logrotate[583]: log does not need rotating (log has already been rotated) gitlab # [ 7.257587] logrotate[583]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed gitlab # [ 7.258542] logrotate[583]: switching euid from 0 to 165 and egid from 0 to 165 (pid 583) gitlab # [ 7.259067] logrotate[583]: considering log /var/gitlab/state/log/*.log gitlab # [ 7.259540] logrotate[583]: log /var/gitlab/state/log/*.log does not exist -- skipping gitlab # [ 7.261442] logrotate[583]: Creating new state gitlab # [ 7.262551] logrotate[583]: switching euid from 165 to 0 and egid from 165 to 0 (pid 583) gitlab # [ 7.262973] logrotate[583]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed gitlab # [ 7.263606] logrotate[583]: switching euid from 0 to 60 and egid from 0 to 60 (pid 583) gitlab # [ 7.264085] logrotate[583]: considering log /var/log/nginx/*.log gitlab # [ 7.264592] logrotate[583]: log /var/log/nginx/*.log does not exist -- skipping gitlab # [ 7.265056] logrotate[583]: Creating new state gitlab # [ 7.265512] logrotate[583]: not running postrotate script, since no logs were rotated gitlab # [ 7.265977] logrotate[583]: switching euid from 60 to 0 and egid from 60 to 0 (pid 583) gitlab # [ 7.266703] systemd[1]: Finished Logrotate configuration check. gitlab # [ 7.267553] systemd[1]: Finished Post-Boot Actions. gitlab: Guest shell says: b'Spawning backdoor root shell...\n' gitlab: connected to guest root shell gitlab # [ 7.291985] systemd[1]: Finished Import lastlog data into lastlog2 database. gitlab: (connecting took 7.92 seconds) gitlab: (finished: waiting for the VM to finish booting, in 7.92 seconds) gitlab # [ 7.308188] nsncd[584]: Oct 10 10:44:44.599 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.308782] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.309487] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.310058] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.312840] systemd[1]: Starting User Login Management... gitlab # [ 7.336284] systemd[1]: Stopped target Host and Network Name Lookups. gitlab # [ 7.337220] systemd[1]: Stopping Host and Network Name Lookups... gitlab # [ 7.339916] systemd[1]: Stopped target User and Group Name Lookups. gitlab # [ 7.340934] systemd[1]: Stopping User and Group Name Lookups... gitlab # [ 7.341910] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... gitlab # [ 7.342768] systemd[1]: nscd.service: Deactivated successfully. gitlab # [ 7.343661] systemd[1]: Stopped Name Service Cache Daemon (nsncd). gitlab # [ 7.344992] systemd[1]: nscd.service: Consumed 20ms CPU time over 118ms wall clock time, 7.3M memory peak, 5.5M read from disk. gitlab # [ 7.348151] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.357147] systemd-logind[633]: New seat seat0. gitlab # [ 7.359191] systemd-logind[633]: Watching system buttons on /dev/input/event2 (Power Button) gitlab # [ 7.359884] systemd-logind[633]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) gitlab # [ 7.360538] systemd-logind[633]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) gitlab # [ 7.363327] systemd[1]: Started User Login Management. gitlab # [ 7.365862] systemd[1]: Starting linger-users.service... gitlab # [ 7.372538] nsncd[674]: Oct 10 10:44:44.664 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.373332] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.376208] systemd[1]: Finished resolvconf update. gitlab # [ 7.377539] systemd[1]: Reached target Preparation for Network. gitlab # [ 7.378269] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.378702] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.381756] systemd[1]: Starting DHCP Client... gitlab # [ 7.385212] systemd[1]: Starting Address configuration of eth1... gitlab # [ 7.387781] systemd[1]: Starting Extra networking commands.... gitlab # [ 7.389213] systemd[1]: linger-users.service: Deactivated successfully. gitlab # [ 7.393074] systemd[1]: Finished linger-users.service. gitlab # [ 7.410897] network-addresses-eth1-start[700]: adding address 192.168.1.1/24... done gitlab # [ 7.413927] network-addresses-eth1-start[700]: adding address 2001:db8:1::1/64... done gitlab # [ 7.420779] systemd[1]: Finished Address configuration of eth1. gitlab # [ 7.453123] systemd[1]: Finished Extra networking commands.. gitlab # [ 7.453966] systemd[1]: Reached target Network. gitlab # [ 7.457317] systemd[1]: Starting gitlab-workhorse.service... gitlab # [ 7.459277] systemd[1]: Starting Nginx Web Server... gitlab # [ 7.461816] systemd[1]: Starting PostgreSQL Server... gitlab # [ 7.464274] dhcpcd[719]: dhcpcd-10.3.2 starting gitlab # [ 7.465283] dhcpcd[769]: dev: loaded udev gitlab # [ 7.466281] systemd[1]: Starting Redis Server - redis-gitlab... gitlab # [ 7.468506] systemd[1]: Starting Permit User Sessions... gitlab # [ 7.622376] 8021q: 802.1Q VLAN Support v1.8 gitlab # [ 7.622880] 8021q: adding VLAN 0 to HW filter on device eth1 gitlab # [ 7.491844] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. gitlab # [ 7.493236] systemd[1]: Finished Permit User Sessions. gitlab # [ 7.497640] systemd[1]: Started Getty on tty1. gitlab # [ 7.500055] systemd[1]: Reached target Login Prompts. gitlab # [ 7.728469] cfg80211: Loading compiled-in X.509 certificates for regulatory database gitlab # [ 7.733188] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' gitlab # [ 7.734092] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' gitlab # [ 7.739003] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 gitlab # [ 7.740045] cfg80211: failed to load regulatory.db gitlab # [ 7.614479] redis-server[790]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. gitlab # [ 7.615262] redis-server[790]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo gitlab # [ 7.615902] redis-server[790]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=790, just started gitlab # [ 7.616553] redis-server[790]: Configuration loaded gitlab # [ 7.618667] redis[790]: monotonic clock: POSIX clock_gettime gitlab # [ 7.619970] systemd[1]: Started Redis Server - redis-gitlab. gitlab # [ 7.620648] redis[790]: Running mode=standalone, port=0. gitlab # [ 7.621176] redis[790]: Server initialized gitlab # [ 7.622054] redis[790]: Ready to accept connections unix gitlab # [ 7.622457] redis[790]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. gitlab # [ 7.774030] 8021q: adding VLAN 0 to HW filter on device eth0 gitlab # [ 7.628125] dhcpcd[769]: eth0: waiting for carrier gitlab # [ 7.628779] dhcpcd[769]: eth0: carrier acquired gitlab # [ 7.632089] nginx-pre-start[792]: nginx: the configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf syntax is ok gitlab # [ 7.632828] nginx-pre-start[792]: nginx: configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf test is successful gitlab # [ 7.633545] dhcpcd[769]: DUID 00:01:00:01:32:5c:d3:9c:52:54:00:12:34:56 gitlab # [ 7.633958] dhcpcd[769]: eth0: IAID 00:12:34:56 gitlab # [ 7.638236] dhcpcd[769]: eth0: adding address fe80::5054:ff:fe12:3456 gitlab # [ 7.639618] postgresql-pre-start[795]: The files belonging to this database system will be owned by user "postgres". gitlab # [ 7.640188] postgresql-pre-start[795]: This user must also own the server process. gitlab # [ 7.640750] systemd[1]: Started Nginx Web Server. gitlab # [ 7.643145] postgresql-pre-start[795]: The database cluster will be initialized with locale "en_US.UTF-8". gitlab # [ 7.643723] postgresql-pre-start[795]: The default database encoding has accordingly been set to "UTF8". gitlab # [ 7.644211] postgresql-pre-start[795]: The default text search configuration will be set to "english". gitlab # [ 7.644752] postgresql-pre-start[795]: Data page checksums are enabled. gitlab # [ 7.645447] postgresql-pre-start[795]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok gitlab # [ 7.645945] postgresql-pre-start[795]: creating subdirectories ... ok gitlab # [ 7.646828] postgresql-pre-start[795]: selecting dynamic shared memory implementation ... posix gitlab # [ 7.692630] postgresql-pre-start[795]: selecting default "max_connections" ... 100 gitlab # [ 7.744603] postgresql-pre-start[795]: selecting default "shared_buffers" ... 128MB gitlab # [ 7.854461] dhcpcd[769]: eth0: soliciting a DHCP lease gitlab # [ 8.007452] NET: Registered PF_PACKET protocol family gitlab # [ 7.868572] dhcpcd[769]: eth0: offered 10.0.2.15 from 10.0.2.2 gitlab # [ 7.875247] dhcpcd[769]: eth0: probing address 10.0.2.15/24 gitlab # [ 7.972316] systemd[1]: Started gitlab-workhorse.service. gitlab # [ 8.125330] postgresql-pre-start[795]: selecting default time zone ... UTC gitlab # [ 8.129801] postgresql-pre-start[795]: creating configuration files ... ok gitlab # [ 8.198736] gitlab-workhorse[827]: time="2026-10-10T10:44:45Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4 gitlab # [ 8.199252] gitlab-workhorse[827]: time="2026-10-10T10:44:45Z" level=info msg="Using redis/go-redis" gitlab # [ 8.199858] gitlab-workhorse[827]: time="2026-10-10T10:44:45Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix gitlab # [ 8.294298] postgresql-pre-start[795]: running bootstrap script ... ok gitlab # [ 8.738299] postgresql-pre-start[795]: performing post-bootstrap initialization ... ok gitlab # [ 9.035568] postgresql-pre-start[795]: syncing data to disk ... ok gitlab # [ 9.037135] postgresql-pre-start[795]: initdb: warning: enabling "trust" authentication for local connections gitlab # [ 9.038157] postgresql-pre-start[795]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. gitlab # [ 9.039276] postgresql-pre-start[795]: Success. You can now start the database server using: gitlab # [ 9.040432] postgresql-pre-start[795]: pg_ctl -D /var/lib/postgresql/18 -l logfile start gitlab # [ 9.149076] postgres[851]: [851] LOG: starting PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit gitlab # [ 9.150810] postgres[851]: [851] LOG: listening on IPv6 address "::1", port 5432 gitlab # [ 9.151970] postgres[851]: [851] LOG: listening on IPv4 address "127.0.0.1", port 5432 gitlab # [ 9.153953] postgres[851]: [851] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" gitlab # [ 9.160751] postgres[866]: [866] LOG: database system was shut down at 2026-10-10 10:44:46 GMT gitlab # [ 9.165285] postgres[851]: [851] LOG: database system is ready to accept connections gitlab # [ 9.168186] systemd[1]: Started PostgreSQL Server. gitlab # [ 9.170238] systemd[1]: Starting PostgreSQL Setup Scripts... gitlab # [ 9.274369] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[856]: mkdir -p /var/gitlab/state/home/.ssh: OK gitlab # [ 9.277479] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[856]: chmod 700 /var/gitlab/state/home/.ssh: OK gitlab # [ 9.334099] postgresql-setup-start[877]: CREATE DATABASE gitlab # [ 9.336115] systemd[1]: Finished gitlab-config.service. gitlab # [ 9.338711] systemd[1]: Started gitaly.service. gitlab # [ 9.356663] postgresql-setup-start[895]: CREATE ROLE gitlab # [ 9.365245] postgresql-setup-start[902]: ALTER DATABASE gitlab # [ 9.384354] postgresql-setup-start[911]: CREATE ROLE gitlab # [ 9.387533] systemd[1]: Finished PostgreSQL Setup Scripts. gitlab # [ 9.388233] systemd[1]: Reached target PostgreSQL. gitlab # [ 9.389338] systemd[1]: Starting gitlab-db-config.service... gitlab # [ 9.390440] systemd[1]: Starting gitlab-postgresql.service... gitlab # [ 9.471766] gitlab-postgresql-start[922]: CREATE DATABASE gitlab # [ 9.501324] gitlab-postgresql-start[927]: CREATE EXTENSION gitlab # [ 9.577043] gitlab-postgresql-start[929]: CREATE EXTENSION gitlab # [ 9.580095] systemd[1]: Finished gitlab-postgresql.service. gitlab # [ 9.659194] dhcpcd[769]: eth0: soliciting an IPv6 router gitlab # [ 9.660183] dhcpcd[769]: eth0: Router Advertisement from fe80::2 gitlab # [ 9.661075] dhcpcd[769]: eth0: adding address fec0::5054:ff:fe12:3456/64 gitlab # [ 9.662044] dhcpcd[769]: eth0: adding route to fec0::/64 gitlab # [ 9.662838] dhcpcd[769]: eth0: adding default route via fe80::2 gitlab # [ 9.871344] gitaly[893]: time="2026-10-10T10:44:47.163Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/ymnlr20kjfmhkzg7i9hp25h3ii8qnn0d-gitaly.toml' and fix the errors that are printed." pid=893 gitlab # [ 9.872137] gitaly[893]: time="2026-10-10T10:44:47.163Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=893 gitlab # [ 9.872639] gitaly[893]: time="2026-10-10T10:44:47.163Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=893 gitlab # [ 9.874203] gitaly[893]: time="2026-10-10T10:44:47.166Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=893 gitlab # [ 9.874929] gitaly[893]: time="2026-10-10T10:44:47.166Z" level=info msg="Starting Gitaly" pid=893 version= gitlab # [ 10.077602] gitaly[893]: time="2026-10-10T10:44:47.369Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=202 pid=893 gitlab # [ 10.078626] gitaly[893]: time="2026-10-10T10:44:47.370Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=893 gitlab # [ 10.079437] gitaly[893]: time="2026-10-10T10:44:47.370Z" level=info msg="finished initializing command factory" duration_ms=0 pid=893 gitlab # [ 10.079882] gitaly[893]: time="2026-10-10T10:44:47.370Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-893/git-exec-566591024.d/git pid=893 gitlab # [ 10.084338] gitaly[893]: time="2026-10-10T10:44:47.376Z" level=info msg="finished detecting git version" duration_ms=6 pid=893 gitlab # [ 10.085152] gitaly[893]: time="2026-10-10T10:44:47.376Z" level=info msg="using Git version" pid=893 version=2.53.g gitlab # [ 10.086500] gitaly[893]: time="2026-10-10T10:44:47.377Z" level=info msg="clearing disk cache object folder" pid=893 storage=default gitlab # [ 10.087498] gitaly[893]: time="2026-10-10T10:44:47.377Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1087232028 pid=893 storage=default gitlab # [ 10.088108] gitaly[893]: time="2026-10-10T10:44:47.377Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1087232028 pid=893 storage=default gitlab # [ 10.088729] gitaly[893]: time="2026-10-10T10:44:47.377Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=893 gitlab # [ 10.089169] gitaly[893]: time="2026-10-10T10:44:47.377Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=893 gitlab # [ 10.089605] gitaly[893]: time="2026-10-10T10:44:47.378Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1087232028 clear_duration_ms=0 pid=893 storage=default gitlab # [ 10.120349] gitaly[893]: time="2026-10-10T10:44:47.412Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=893 gitlab # [ 10.122616] gitaly[893]: time="2026-10-10T10:44:47.412Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=893 schema=unix gitlab # [ 10.123749] gitaly[893]: time="2026-10-10T10:44:47.413Z" level=info msg="counting repositories" pid=893 gitlab # [ 10.124925] gitaly[893]: time="2026-10-10T10:44:47.414Z" level=info msg="starting to count repositories" pid=893 storage_path=/var/gitlab/state/repositories gitlab # [ 10.125831] gitaly[893]: time="2026-10-10T10:44:47.414Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=893 storage_path=/var/gitlab/state/repositories gitlab # [ 10.126802] gitaly[893]: time="2026-10-10T10:44:47.414Z" level=info msg="completed counting all repositories" counting_duration=0 pid=893 gitlab # [ 10.127680] gitaly[893]: time="2026-10-10T10:44:47.414Z" level=info msg="finished tempdir cleaner walk" pid=893 storage=default time_ms=0 gitlab # [ 10.128860] gitaly[893]: time="2026-10-10T10:44:47.415Z" level=info msg="listening at address" address=/run/gitaly/gitaly-893/sock.d/intern pid=893 schema=unix gitlab # [ 10.129840] gitaly[893]: time="2026-10-10T10:44:47.416Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=893 gitlab # [ 10.130666] gitaly[893]: time="2026-10-10T10:44:47.416Z" level=info msg="Started Gitaly" duration_ms=249 pid=893 gitlab # [ 10.131556] gitaly[893]: time="2026-10-10T10:44:47.416Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=893 gitlab # [ 10.132455] gitaly[893]: time="2026-10-10T10:44:47.416Z" level=info msg="maintenance: daily scheduled" pid=893 scheduled="2026-10-10 12:00:00 +0000 UTC" gitlab # [ 10.921917] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[917]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 10.923285] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[917]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 10.952578] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[917]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 12.001390] gitaly[893]: time="2026-10-10T10:44:49.292Z" level=info msg="License database preloaded" duration_ms=2128 pid=893 gitlab # [ 12.247217] dhcpcd[769]: eth0: leased 10.0.2.15 for 86400 seconds gitlab # [ 12.249157] dhcpcd[769]: eth0: adding route to 10.0.2.0/24 gitlab # [ 12.251143] dhcpcd[769]: eth0: adding default route via 10.0.2.2 gitlab # [ 12.375096] systemd[1]: Started DHCP Client. gitlab # [ 12.378088] systemd[1]: Reached target Network is Online. gitlab # [ 41.290247] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[917]: W, [2026-10-10T10:45:18.581788 #917] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 43.079205] postgres[1321]: [1321] ERROR: relation "feature_gates" does not exist at character 523 gitlab # [ 43.080429] postgres[1321]: [1321] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), gitlab # [ 43.081062] postgres[1321]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, gitlab # [ 43.082136] postgres[1321]: c.collname, col_description(a.attrelid, a.attnum) AS comment, gitlab # [ 43.083235] postgres[1321]: attidentity AS identity, gitlab # [ 43.084329] postgres[1321]: attgenerated as attgenerated gitlab # [ 43.085409] postgres[1321]: FROM pg_attribute a gitlab # [ 43.086868] postgres[1321]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum gitlab # [ 43.087975] postgres[1321]: LEFT JOIN pg_type t ON a.atttypid = t.oid gitlab # [ 43.089445] postgres[1321]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation gitlab # [ 43.090901] postgres[1321]: WHERE a.attrelid = '"feature_gates"'::regclass gitlab # [ 43.092021] postgres[1321]: AND a.attnum > 0 AND NOT a.attisdropped gitlab # [ 43.093112] postgres[1321]: ORDER BY a.attnum gitlab # [ 43.094218] postgres[1321]: gitlab # [ 46.408192] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1354]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping gitlab # [ 46.409417] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1354]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping gitlab # [ 48.250158] gitlab-workhorse[827]: time="2026-10-10T10:45:25Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 58.259436] gitlab-workhorse[827]: time="2026-10-10T10:45:35Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 61.322764] postgres[1420]: [1420] ERROR: relation "table_without_models" does not exist gitlab # [ 61.324361] postgres[1420]: [1420] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 68.271296] gitlab-workhorse[827]: time="2026-10-10T10:45:45Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 72.965534] systemd[1]: Finished gitlab-db-config.service. gitlab # [ 72.967759] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 72.969677] systemd[1]: Starting gitlab.service... gitlab # [ 73.331988] bundle[1587]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 73.333308] bundle[1587]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 73.337144] bundle[1587]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 76.110572] sidekiq-cluster[1586]: {"severity":"INFO","time":"2026-10-10T10:45:53.385Z","message":"Starting cluster with 1 processes"} gitlab # [ 76.188261] bundle[1587]: {"timestamp":"2026-10-10T10:45:53.462Z","pid":1587,"message":"* Enabling systemd notification integration"} gitlab # [ 76.189604] bundle[1587]: {"timestamp":"2026-10-10T10:45:53.480Z","pid":1587,"message":"Puma starting in cluster mode..."} gitlab # [ 76.190667] bundle[1587]: {"timestamp":"2026-10-10T10:45:53.480Z","pid":1587,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"} gitlab # [ 76.191607] bundle[1587]: {"timestamp":"2026-10-10T10:45:53.480Z","pid":1587,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 76.192601] bundle[1587]: {"timestamp":"2026-10-10T10:45:53.480Z","pid":1587,"message":"* Min threads: 0"} gitlab # [ 76.193477] bundle[1587]: {"timestamp":"2026-10-10T10:45:53.480Z","pid":1587,"message":"* Max threads: 4"} gitlab # [ 76.194262] bundle[1587]: {"timestamp":"2026-10-10T10:45:53.480Z","pid":1587,"message":"* Environment: production"} gitlab # [ 76.194987] bundle[1587]: {"timestamp":"2026-10-10T10:45:53.480Z","pid":1587,"message":"* Master PID: 1587"} gitlab # [ 76.195676] bundle[1587]: {"timestamp":"2026-10-10T10:45:53.480Z","pid":1587,"message":"* Workers: 2"} gitlab # [ 76.196297] bundle[1587]: {"timestamp":"2026-10-10T10:45:53.480Z","pid":1587,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} gitlab # [ 76.197052] bundle[1587]: {"timestamp":"2026-10-10T10:45:53.480Z","pid":1587,"message":"* Preloading application"} gitlab # [ 78.280112] gitlab-workhorse[827]: time="2026-10-10T10:45:55Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 88.281124] gitlab-workhorse[827]: time="2026-10-10T10:46:05Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 98.291353] gitlab-workhorse[827]: time="2026-10-10T10:46:15Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 98.752403] bundle[1587]: W, [2026-10-10T10:46:16.044011 #1587] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 101.118163] sidekiq-cluster[1617]: 2026-10-10T10:46:18.409Z pid=1617 tid=wl5 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1617, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>434229248} gitlab # [ 101.287390] sidekiq-cluster[1617]: W, [2026-10-10T10:46:18.579650 #1617] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 101.326140] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:18.618Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 103.290876] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.582Z","message":"GitLab reliable fetch activated!"} gitlab # [ 103.292187] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.582Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 103.312064] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.604Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} gitlab # [ 103.313928] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.606Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"} gitlab # [ 103.316263] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.608Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} gitlab # [ 103.318400] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.610Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} gitlab # [ 103.320734] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.613Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} gitlab # [ 103.322978] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.615Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} gitlab # [ 103.325118] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.617Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} gitlab # [ 103.326886] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.619Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} gitlab # [ 103.328711] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.621Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} gitlab # [ 103.330570] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.622Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} gitlab # [ 103.332363] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.624Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} gitlab # [ 103.334601] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.626Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} gitlab # [ 103.336375] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.628Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} gitlab # [ 103.338312] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.630Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} gitlab # [ 103.340172] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.632Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} gitlab # [ 103.342105] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.634Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} gitlab # [ 103.344583] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.636Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} gitlab # [ 103.346869] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.639Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} gitlab # [ 103.349315] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.641Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} gitlab # [ 103.351714] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.644Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} gitlab # [ 103.354092] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.646Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} gitlab # [ 103.356467] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.648Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} gitlab # [ 103.358844] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.651Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} gitlab # [ 103.360945] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.653Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} gitlab # [ 103.363132] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.655Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} gitlab # [ 103.365938] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.658Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} gitlab # [ 103.368366] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.660Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} gitlab # [ 103.375726] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.667Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} gitlab # [ 103.382677] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.674Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} gitlab # [ 103.389375] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.681Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} gitlab # [ 103.394877] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.687Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} gitlab # [ 103.401350] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.693Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} gitlab # [ 103.407806] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.699Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} gitlab # [ 103.414283] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.706Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} gitlab # [ 103.420243] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.712Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} gitlab # [ 103.425887] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.718Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} gitlab # [ 103.431962] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.724Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} gitlab # [ 103.437793] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.730Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} gitlab # [ 103.442987] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.735Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} gitlab # [ 103.448295] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.740Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} gitlab # [ 103.454449] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.746Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} gitlab # [ 103.459845] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.752Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} gitlab # [ 103.466328] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.758Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} gitlab # [ 103.472118] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.764Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} gitlab # [ 103.479064] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.771Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} gitlab # [ 103.485579] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.777Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} gitlab # [ 103.491210] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.783Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} gitlab # [ 103.497104] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.789Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} gitlab # [ 103.503099] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.795Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} gitlab # [ 103.508123] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.800Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} gitlab # [ 103.513728] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.806Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} gitlab # [ 103.519786] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.812Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} gitlab # [ 103.535847] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.828Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} gitlab # [ 103.544423] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.836Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} gitlab # [ 103.549712] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.842Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} gitlab # [ 103.554581] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.846Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} gitlab # [ 103.559417] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.851Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} gitlab # [ 103.564790] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.857Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} gitlab # [ 103.569268] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.861Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} gitlab # [ 103.574185] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.866Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} gitlab # [ 103.579038] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.871Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} gitlab # [ 103.583094] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.875Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} gitlab # [ 103.588376] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.880Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} gitlab # [ 103.593899] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.886Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} gitlab # [ 103.597991] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.890Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} gitlab # [ 103.602871] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.895Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} gitlab # [ 103.609877] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.902Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} gitlab # [ 103.650989] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.943Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} gitlab # [ 103.656557] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.948Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} gitlab # [ 103.662831] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.955Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} gitlab # [ 103.669879] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.962Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} gitlab # [ 103.674530] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.966Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} gitlab # [ 103.678758] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.971Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} gitlab # [ 103.682994] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.975Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} gitlab # [ 103.685844] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.978Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} gitlab # [ 103.687886] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.980Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} gitlab # [ 103.690644] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.983Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} gitlab # [ 103.692894] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.985Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} gitlab # [ 103.694967] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.987Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} gitlab # [ 103.697367] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.989Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} gitlab # [ 103.699283] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.991Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} gitlab # [ 103.701760] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.994Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} gitlab # [ 103.704396] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.996Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} gitlab # [ 103.706821] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:20.999Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} gitlab # [ 103.708825] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:21.001Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} gitlab # [ 103.711520] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:21.003Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} gitlab # [ 103.713903] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:21.006Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} gitlab # [ 103.716385] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:21.008Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} gitlab # [ 103.718324] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:21.010Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} gitlab # [ 103.720694] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:21.013Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} gitlab # [ 103.722836] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:21.015Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} gitlab # [ 103.725122] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:21.017Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} gitlab # [ 103.727029] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:21.019Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} gitlab # [ 103.729073] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:21.021Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} gitlab # [ 103.731499] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:21.023Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} gitlab # [ 103.734185] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:21.026Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} gitlab # [ 103.736184] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:21.028Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} gitlab # [ 103.738618] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:21.031Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} gitlab # [ 103.746265] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:21.038Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} gitlab # [ 103.748222] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:21.040Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} gitlab # [ 103.750310] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:21.042Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} gitlab # [ 103.753107] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:21.045Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} gitlab # [ 103.755912] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:21.048Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} gitlab # [ 103.758271] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:21.050Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} gitlab # [ 103.761090] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:21.053Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} gitlab # [ 103.762974] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:21.055Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} gitlab # [ 103.765357] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:21.057Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} gitlab # [ 103.767764] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:21.060Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} gitlab # [ 103.770099] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:21.062Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} gitlab # [ 108.301426] gitlab-workhorse[827]: time="2026-10-10T10:46:25Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 118.313861] gitlab-workhorse[827]: time="2026-10-10T10:46:35Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 128.313744] gitlab-workhorse[827]: time="2026-10-10T10:46:45Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 128.681169] postgres[2064]: [2064] ERROR: relation "table_without_models" does not exist gitlab # [ 128.682563] postgres[2064]: [2064] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 129.683813] postgres[2084]: [2084] ERROR: relation "table_without_models" does not exist gitlab # [ 129.684647] postgres[2084]: [2084] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 131.276677] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:48.568Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 131.281882] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:48.573Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 131.282556] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:48.574Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 131.283711] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:48.574Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 131.296281] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:48.587Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 132.459639] bundle[1587]: {"timestamp":"2026-10-10T10:46:49.751Z","pid":1587,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} gitlab # [ 132.460678] bundle[1587]: {"timestamp":"2026-10-10T10:46:49.751Z","pid":1587,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} gitlab # [ 132.461751] bundle[1587]: {"timestamp":"2026-10-10T10:46:49.751Z","pid":1587,"message":"! #\u003cThread:0x000075d4a5653520 /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} gitlab # [ 132.462765] bundle[1587]: {"timestamp":"2026-10-10T10:46:49.751Z","pid":1587,"message":"! #\u003cThread:0x000075d4a58bdef0 /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} gitlab # [ 132.463622] bundle[1587]: {"timestamp":"2026-10-10T10:46:49.751Z","pid":1587,"message":"Use Ctrl-C to stop"} gitlab # [ 132.464472] bundle[1587]: {"timestamp":"2026-10-10T10:46:49.751Z","pid":1587,"message":"Sending status to systemd every 1.0 sec"} gitlab # [ 132.651539] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:49.943Z","message":"Cleaning working queues"} gitlab # [ 132.656785] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:49.948Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"c7f8d74b2b97e49054cc78f9","created_at":"2026-10-10T10:45:49.946Z","trace_propagation_headers":{"sentry-trace":"9c32d35d133f403a9768115b165878fe-394d4a4b58b4470f","baggage":"sentry-trace_id=9c32d35d133f403a9768115b165878fe,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"9f602136a857cc5a926a969e6078eeb2","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-10T10:45:49.953Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-c7f8d74b2b97e49054cc78f9: start","job_status":"start","queue_duration_s":59.994848,"scheduling_latency_s":59.994911} gitlab # [ 132.801822] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:46:50.093Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"c7f8d74b2b97e49054cc78f9","created_at":"2026-10-10T10:45:49.946Z","trace_propagation_headers":{"sentry-trace":"9c32d35d133f403a9768115b165878fe-394d4a4b58b4470f","baggage":"sentry-trace_id=9c32d35d133f403a9768115b165878fe,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"9f602136a857cc5a926a969e6078eeb2","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-10T10:45:49.953Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-c7f8d74b2b97e49054cc78f9: done: 0.145292 sec","job_status":"done","queue_duration_s":59.994848,"scheduling_latency_s":59.994911,"redis_calls":26,"redis_duration_s":0.00397,"redis_read_bytes":914,"redis_write_bytes":2887,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.002182,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001076,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000712,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.056736,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.082511324,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.145292,"completed_at":"2026-10-10T10:46:50.093Z","load_balancing_strategy":"primary","db_duration_s":0.051619,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 132.861270] bundle[1587]: {"timestamp":"2026-10-10T10:46:50.098Z","pid":1587,"message":"- Worker 1 (PID: 2173) booted in 0.29s, phase: 0"} gitlab # [ 132.862795] bundle[1587]: {"timestamp":"2026-10-10T10:46:50.101Z","pid":1587,"message":"- Worker 0 (PID: 2169) booted in 0.31s, phase: 0"} gitlab # [ 132.864498] systemd[1]: Started gitlab.service. gitlab # [ 132.865258] systemd[1]: Reached target Common target for all GitLab services.. gitlab # [ 132.866437] systemd[1]: Starting setup-gitlab.service... gitlab # [ 132.870306] setup-gitlab-start[2204]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' gitlab # [ 135.316784] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:46:52 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 2424 gitlab # [ 135.320774] setup-gitlab-start[2204]: + TOKEN=nixbot-test-token-of-20-chars gitlab # [ 135.321899] setup-gitlab-start[2204]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n' gitlab # [ 135.335570] runuser[2226]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) gitlab # [ 135.700158] setup-gitlab-start[2227]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 135.705163] setup-gitlab-start[2227]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 135.709863] setup-gitlab-start[2227]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 136.272393] setup-gitlab-start[2227]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 136.274343] setup-gitlab-start[2227]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 136.275386] setup-gitlab-start[2227]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 147.328921] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:04.620Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"8be1a95e9ff3f7e607a26de0","created_at":"2026-10-10T10:47:04.595Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2cfe0a2bcc474ee7b0ef65b2f771ded","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:04.619Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-8be1a95e9ff3f7e607a26de0: start","job_status":"start","queue_duration_s":0.001191,"scheduling_latency_s":0.001216} gitlab # [ 147.363360] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:04.655Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 147.374531] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:04.666Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"8be1a95e9ff3f7e607a26de0","created_at":"2026-10-10T10:47:04.595Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2cfe0a2bcc474ee7b0ef65b2f771ded","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:04.619Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-8be1a95e9ff3f7e607a26de0: done: 0.045912 sec","job_status":"done","queue_duration_s":0.001191,"scheduling_latency_s":0.001216,"redis_calls":10,"redis_duration_s":0.006734,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002389,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003481,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000864,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007993,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000608178,"duration_s":0.045912,"completed_at":"2026-10-10T10:47:04.666Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000376,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.375879] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:04.667Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"d77e3dc7fd1c042292d3f9ae","created_at":"2026-10-10T10:47:04.642Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"56110b5513d14171a09848922dfb40e1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-10T10:47:04.649Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-d77e3dc7fd1c042292d3f9ae: start","job_status":"start","queue_duration_s":0.018801,"scheduling_latency_s":0.018818} gitlab # [ 147.401071] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:04.693Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"d77e3dc7fd1c042292d3f9ae","created_at":"2026-10-10T10:47:04.642Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"56110b5513d14171a09848922dfb40e1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-10T10:47:04.649Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-d77e3dc7fd1c042292d3f9ae: done: 0.025297 sec","job_status":"done","queue_duration_s":0.018801,"scheduling_latency_s":0.018818,"redis_calls":10,"redis_duration_s":0.001699,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000559,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000861,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000279,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016088,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026819,"duration_s":0.025297,"completed_at":"2026-10-10T10:47:04.693Z","load_balancing_strategy":"primary","db_duration_s":0.002474,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.470610] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:04.762Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"c29a8a3245df755fb546d12f","created_at":"2026-10-10T10:47:04.659Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7b41555cdab6ba6480a199e679625b89","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:04.761Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-c29a8a3245df755fb546d12f: start","job_status":"start","queue_duration_s":0.001529,"scheduling_latency_s":0.001571} gitlab # [ 147.516865] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:04.809Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 147.533861] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:04.825Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"c29a8a3245df755fb546d12f","created_at":"2026-10-10T10:47:04.659Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7b41555cdab6ba6480a199e679625b89","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:04.761Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-c29a8a3245df755fb546d12f: done: 0.063264 sec","job_status":"done","queue_duration_s":0.001529,"scheduling_latency_s":0.001571,"redis_calls":10,"redis_duration_s":0.033069,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000498,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004948,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.027623,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014637,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000672991,"duration_s":0.063264,"completed_at":"2026-10-10T10:47:04.825Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001585,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.554989] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:04.828Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"716bc715d3813f19d463c726","created_at":"2026-10-10T10:47:04.799Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cadd15edc4c9078bbf9f10fa8f31bc9e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-10T10:47:04.805Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-716bc715d3813f19d463c726: start","job_status":"start","queue_duration_s":0.023487,"scheduling_latency_s":0.023519} gitlab # [ 147.569192] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:04.861Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 147.587953] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:04.879Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"716bc715d3813f19d463c726","created_at":"2026-10-10T10:47:04.799Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cadd15edc4c9078bbf9f10fa8f31bc9e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-10T10:47:04.805Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-716bc715d3813f19d463c726: done: 0.050817 sec","job_status":"done","queue_duration_s":0.023487,"scheduling_latency_s":0.023519,"redis_calls":10,"redis_duration_s":0.006079,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000782,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004792,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000505,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015164,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000908775,"duration_s":0.050817,"completed_at":"2026-10-10T10:47:04.879Z","load_balancing_strategy":"primary","db_duration_s":0.014125,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.589848] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:04.882Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"ed3c840e166a5f4ac60e2162","created_at":"2026-10-10T10:47:04.822Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd2d847e20183249684f273c60376078","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:04.831Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-ed3c840e166a5f4ac60e2162: start","job_status":"start","queue_duration_s":0.050582,"scheduling_latency_s":0.050623} gitlab # [ 147.616752] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:04.908Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"ed3c840e166a5f4ac60e2162","created_at":"2026-10-10T10:47:04.822Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd2d847e20183249684f273c60376078","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:04.831Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-ed3c840e166a5f4ac60e2162: done: 0.026757 sec","job_status":"done","queue_duration_s":0.050582,"scheduling_latency_s":0.050623,"redis_calls":13,"redis_duration_s":0.002222,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000392,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001144,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000686,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016178,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.006888027,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.026757,"completed_at":"2026-10-10T10:47:04.908Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002489,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 147.656710] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:04.910Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"ad5dfac7bb611e2c6a4a8038","created_at":"2026-10-10T10:47:04.860Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"584a3d0a01a78368223fb793589986e6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:04.869Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-ad5dfac7bb611e2c6a4a8038: start","job_status":"start","queue_duration_s":0.04092,"scheduling_latency_s":0.040934} gitlab # [ 147.657489] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:04.923Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"ad5dfac7bb611e2c6a4a8038","created_at":"2026-10-10T10:47:04.860Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"584a3d0a01a78368223fb793589986e6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:04.869Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-ad5dfac7bb611e2c6a4a8038: done: 0.013906 sec","job_status":"done","queue_duration_s":0.04092,"scheduling_latency_s":0.040934,"redis_calls":12,"redis_duration_s":0.001519,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000492,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000738,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000289,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00808,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000206172,"duration_s":0.013906,"completed_at":"2026-10-10T10:47:04.923Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000459,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.704774] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:04.967Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"3b4452974e56db5fa61dc178","created_at":"2026-10-10T10:47:04.961Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3817387a7cedd895048e5e163513404f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:04.965Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-3b4452974e56db5fa61dc178: start","job_status":"start","queue_duration_s":0.001063,"scheduling_latency_s":0.001086} gitlab # [ 147.770938] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:05.062Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"3b4452974e56db5fa61dc178","created_at":"2026-10-10T10:47:04.961Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3817387a7cedd895048e5e163513404f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:04.965Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-3b4452974e56db5fa61dc178: done: 0.095825 sec","job_status":"done","queue_duration_s":0.001063,"scheduling_latency_s":0.001086,"redis_calls":12,"redis_duration_s":0.006938000000000001,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001068,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004525,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001345,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029842,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000846756,"exclusive_lock_hold_duration_s":0.057274039000000006,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.095825,"completed_at":"2026-10-10T10:47:05.062Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.044929,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.791656] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:05.068Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"0ddea91108c0b92410d4381d","created_at":"2026-10-10T10:47:04.976Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"14b88cf65b545fd10b7f8bfbf8176f81","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:04.981Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-0ddea91108c0b92410d4381d: start","job_status":"start","queue_duration_s":0.08706,"scheduling_latency_s":0.087076} gitlab # [ 148.225981] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:05.517Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"0ddea91108c0b92410d4381d","created_at":"2026-10-10T10:47:04.976Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"14b88cf65b545fd10b7f8bfbf8176f81","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:04.981Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-0ddea91108c0b92410d4381d: done: 0.449005 sec","job_status":"done","queue_duration_s":0.08706,"scheduling_latency_s":0.087076,"redis_calls":565,"redis_duration_s":0.228154,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000262,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000742,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.22715,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.252401,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000208127,"duration_s":0.449005,"completed_at":"2026-10-10T10:47:05.517Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000332,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.305439] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:05.597Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"2d8fc3c5225e8d0652448f50","created_at":"2026-10-10T10:47:04.995Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.010Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-2d8fc3c5225e8d0652448f50: start","job_status":"start","queue_duration_s":0.587013,"scheduling_latency_s":0.587061} gitlab # [ 148.410606] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:05.701Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"2d8fc3c5225e8d0652448f50","created_at":"2026-10-10T10:47:04.995Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.010Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-2d8fc3c5225e8d0652448f50: done: 0.104706 sec","job_status":"done","queue_duration_s":0.587013,"scheduling_latency_s":0.587061,"redis_calls":71,"redis_duration_s":0.014266000000000001,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.001837,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000831,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.01107,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000528,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.07522,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000296685,"duration_s":0.104706,"completed_at":"2026-10-10T10:47:05.701Z","load_balancing_strategy":"primary","db_duration_s":0.00243,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.413472] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:05.703Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"8509db2a9600fa3f3e217439","created_at":"2026-10-10T10:47:05.022Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eb0a8a11f45062c10452915cc971e9f0","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.027Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-8509db2a9600fa3f3e217439: start","job_status":"start","queue_duration_s":0.675761,"scheduling_latency_s":0.675775} gitlab # [ 148.480863] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:05.728Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"8509db2a9600fa3f3e217439","created_at":"2026-10-10T10:47:05.022Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eb0a8a11f45062c10452915cc971e9f0","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.027Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-8509db2a9600fa3f3e217439: done: 0.025736 sec","job_status":"done","queue_duration_s":0.675761,"scheduling_latency_s":0.675775,"redis_calls":10,"redis_duration_s":0.001355,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000287,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000755,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000313,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010275,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000226844,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.025736,"completed_at":"2026-10-10T10:47:05.728Z","load_balancing_strategy":"primary","db_duration_s":0.010251,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.519479] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:05.729Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"d9cfd4da2bdce6d3b5b5d473","created_at":"2026-10-10T10:47:05.044Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"089983a7f83a4363e88af8716f6e5dd3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.050Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-d9cfd4da2bdce6d3b5b5d473: start","job_status":"start","queue_duration_s":0.67942,"scheduling_latency_s":0.679433} gitlab # [ 148.533051] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:05.825Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"d9cfd4da2bdce6d3b5b5d473","created_at":"2026-10-10T10:47:05.044Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"089983a7f83a4363e88af8716f6e5dd3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.050Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-d9cfd4da2bdce6d3b5b5d473: done: 0.0952 sec","job_status":"done","queue_duration_s":0.67942,"scheduling_latency_s":0.679433,"redis_calls":14,"redis_duration_s":0.001909,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000475,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00075,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000684,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.073,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.072,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017311,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000305905,"exclusive_lock_hold_duration_s":0.080717802,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.0952,"completed_at":"2026-10-10T10:47:05.825Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.073363,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.539110] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:05.826Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"f7e2c8cc8a4d1d1e62e0579c","created_at":"2026-10-10T10:47:05.056Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"995a16a318d129911d265f085e53525b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.065Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-f7e2c8cc8a4d1d1e62e0579c: start","job_status":"start","queue_duration_s":0.761119,"scheduling_latency_s":0.761133} gitlab # [ 148.552033] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:05.834Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 148.555230] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:05.838Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"f7e2c8cc8a4d1d1e62e0579c","created_at":"2026-10-10T10:47:05.056Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"995a16a318d129911d265f085e53525b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.065Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-f7e2c8cc8a4d1d1e62e0579c: done: 0.012169 sec","job_status":"done","queue_duration_s":0.761119,"scheduling_latency_s":0.761133,"redis_calls":10,"redis_duration_s":0.001288,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000266,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000706,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000316,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006546,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000195276,"duration_s":0.012169,"completed_at":"2026-10-10T10:47:05.838Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000869,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.607903] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:05.839Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"4f829b65659793d4dbb72ce1","created_at":"2026-10-10T10:47:05.068Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"839736cddd511002e664318536e59c89","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.081Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-4f829b65659793d4dbb72ce1: start","job_status":"start","queue_duration_s":0.758409,"scheduling_latency_s":0.758423} gitlab # [ 148.677803] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:05.969Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"4f829b65659793d4dbb72ce1","created_at":"2026-10-10T10:47:05.068Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"839736cddd511002e664318536e59c89","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.081Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-4f829b65659793d4dbb72ce1: done: 0.13031 sec","job_status":"done","queue_duration_s":0.758409,"scheduling_latency_s":0.758423,"redis_calls":12,"redis_duration_s":0.002218,"redis_read_bytes":11,"redis_write_bytes":1370,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000266,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000987,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000965,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.02,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.098974,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000303111,"exclusive_lock_hold_duration_s":0.11589880799999999,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.13031,"completed_at":"2026-10-10T10:47:05.969Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.033434,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.719511] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.011Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"c4ae4c18a55371021a811ea6","created_at":"2026-10-10T10:47:05.083Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"63b6b2338ba62aa27827dc9e899ae7c1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.121Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-c4ae4c18a55371021a811ea6: start","job_status":"start","queue_duration_s":0.889715,"scheduling_latency_s":0.889737} gitlab # [ 148.756695] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.048Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"c4ae4c18a55371021a811ea6","created_at":"2026-10-10T10:47:05.083Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"63b6b2338ba62aa27827dc9e899ae7c1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.121Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-c4ae4c18a55371021a811ea6: done: 0.036898 sec","job_status":"done","queue_duration_s":0.889715,"scheduling_latency_s":0.889737,"redis_calls":12,"redis_duration_s":0.001964,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000255,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001026,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000683,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013796,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000360102,"exclusive_lock_hold_duration_s":0.006786338,"duration_s":0.036898,"completed_at":"2026-10-10T10:47:06.048Z","load_balancing_strategy":"primary","db_duration_s":0.004406,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.795433] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.087Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"5ed971fd5b83cd456ee82eb6","created_at":"2026-10-10T10:47:05.132Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"64ca6e871077eacd2fb73b31beea312b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-10T10:47:05.142Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-5ed971fd5b83cd456ee82eb6: start","job_status":"start","queue_duration_s":0.945078,"scheduling_latency_s":0.945117} gitlab # [ 148.814639] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.106Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 148.827393] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.119Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"5ed971fd5b83cd456ee82eb6","created_at":"2026-10-10T10:47:05.132Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"64ca6e871077eacd2fb73b31beea312b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-10T10:47:05.142Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-5ed971fd5b83cd456ee82eb6: done: 0.03191 sec","job_status":"done","queue_duration_s":0.945078,"scheduling_latency_s":0.945117,"redis_calls":10,"redis_duration_s":0.003741,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000653,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002483,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000605,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017367,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000817422,"duration_s":0.03191,"completed_at":"2026-10-10T10:47:06.119Z","load_balancing_strategy":"primary","db_duration_s":0.000603,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.849132] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.121Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"c4db66af6ef45ed03ada04dc","created_at":"2026-10-10T10:47:05.151Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"59250d284538806b426e7adcd53fa964","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.158Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-c4db66af6ef45ed03ada04dc: start","job_status":"start","queue_duration_s":0.96296,"scheduling_latency_s":0.962992} gitlab # [ 148.887912] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.179Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"c4db66af6ef45ed03ada04dc","created_at":"2026-10-10T10:47:05.151Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"59250d284538806b426e7adcd53fa964","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.158Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-c4db66af6ef45ed03ada04dc: done: 0.057738 sec","job_status":"done","queue_duration_s":0.96296,"scheduling_latency_s":0.962992,"redis_calls":10,"redis_duration_s":0.032147,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000586,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.030994,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000567,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01588,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000428826,"duration_s":0.057738,"completed_at":"2026-10-10T10:47:06.179Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000707,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.931669] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.223Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"cccc205d1fea117cdff273e6","created_at":"2026-10-10T10:47:05.170Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9e4a180adcdfbdb58d75da8fce5eda04","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.177Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-cccc205d1fea117cdff273e6: start","job_status":"start","queue_duration_s":1.045746,"scheduling_latency_s":1.045785} gitlab # [ 148.969865] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.261Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"cccc205d1fea117cdff273e6","created_at":"2026-10-10T10:47:05.170Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9e4a180adcdfbdb58d75da8fce5eda04","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.177Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-cccc205d1fea117cdff273e6: done: 0.038151 sec","job_status":"done","queue_duration_s":1.045746,"scheduling_latency_s":1.045785,"redis_calls":10,"redis_duration_s":0.003209,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000672,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001915,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000622,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021057,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000536381,"duration_s":0.038151,"completed_at":"2026-10-10T10:47:06.261Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003114,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.009715] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.301Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"9d0565268da61859e60df3b3","created_at":"2026-10-10T10:47:05.199Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"14265dd8a07157754612ce69164160e4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.208Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-9d0565268da61859e60df3b3: start","job_status":"start","queue_duration_s":1.092652,"scheduling_latency_s":1.092753} gitlab # [ 149.027975] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.320Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 149.040375] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.331Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"9d0565268da61859e60df3b3","created_at":"2026-10-10T10:47:05.199Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"14265dd8a07157754612ce69164160e4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.208Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-9d0565268da61859e60df3b3: done: 0.030383 sec","job_status":"done","queue_duration_s":1.092652,"scheduling_latency_s":1.092753,"redis_calls":10,"redis_duration_s":0.003007,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000589,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00181,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000608,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016336,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000455365,"duration_s":0.030383,"completed_at":"2026-10-10T10:47:06.331Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000686,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.061789] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.334Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"a15ef5a2f6fef8f74c304b36","created_at":"2026-10-10T10:47:05.216Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3353226adb1f3b5534340e9a9c47a1e6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.225Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-a15ef5a2f6fef8f74c304b36: start","job_status":"start","queue_duration_s":1.108843,"scheduling_latency_s":1.108873} gitlab # [ 149.120570] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.412Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"a15ef5a2f6fef8f74c304b36","created_at":"2026-10-10T10:47:05.216Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3353226adb1f3b5534340e9a9c47a1e6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.225Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-a15ef5a2f6fef8f74c304b36: done: 0.077776 sec","job_status":"done","queue_duration_s":1.108843,"scheduling_latency_s":1.108873,"redis_calls":19,"redis_duration_s":0.033504,"redis_read_bytes":22,"redis_write_bytes":4781,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.029856,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000342,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3011,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002443,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000863,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03299,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000829994,"duration_s":0.077776,"completed_at":"2026-10-10T10:47:06.412Z","load_balancing_strategy":"primary","db_duration_s":0.002823,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.161390] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.453Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"fb8449389d4c5f4ae7e26e5c","created_at":"2026-10-10T10:47:05.234Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fca4c3ff811a201f3aea633e83432c73","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.244Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-fb8449389d4c5f4ae7e26e5c: start","job_status":"start","queue_duration_s":1.20883,"scheduling_latency_s":1.208871} gitlab # [ 149.220079] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.511Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"fb8449389d4c5f4ae7e26e5c","created_at":"2026-10-10T10:47:05.234Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fca4c3ff811a201f3aea633e83432c73","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.244Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-fb8449389d4c5f4ae7e26e5c: done: 0.058633 sec","job_status":"done","queue_duration_s":1.20883,"scheduling_latency_s":1.208871,"redis_calls":16,"redis_duration_s":0.005074,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002657,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00185,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000567,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031039,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000494755,"duration_s":0.058633,"completed_at":"2026-10-10T10:47:06.511Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013591,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.248989] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.513Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"e5d437d52d283f09cf58f343","created_at":"2026-10-10T10:47:05.260Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b72e85697c17a786fbd4f09b975172c3","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.271Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-e5d437d52d283f09cf58f343: start","job_status":"start","queue_duration_s":1.24278,"scheduling_latency_s":1.242805} gitlab # [ 149.298133] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.589Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"e5d437d52d283f09cf58f343","created_at":"2026-10-10T10:47:05.260Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-1589fb6c735946de","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b72e85697c17a786fbd4f09b975172c3","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.271Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-e5d437d52d283f09cf58f343: done: 0.075605 sec","job_status":"done","queue_duration_s":1.24278,"scheduling_latency_s":1.242805,"redis_calls":12,"redis_duration_s":0.032285,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000279,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001355,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.030651,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012881,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000348648,"exclusive_lock_hold_duration_s":0.004205843,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.075605,"completed_at":"2026-10-10T10:47:06.589Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0031,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.338437] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.630Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02ded78000cc38112a3a0f47","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02ded78000cc38112a3a0f47: start","job_status":"start","queue_duration_s":0.936409,"scheduling_latency_s":0.936475} gitlab # [ 149.383235] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.674Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02ded78000cc38112a3a0f47","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02ded78000cc38112a3a0f47: done: 0.044495 sec","job_status":"done","queue_duration_s":0.936409,"scheduling_latency_s":0.936475,"redis_calls":16,"redis_duration_s":0.004404,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000598,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001216,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00259,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019577,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000677181,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.044495,"completed_at":"2026-10-10T10:47:06.674Z","load_balancing_strategy":"primary","db_duration_s":0.001536,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.422348] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.714Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3fe56b633922fbb96ffa15aa","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3fe56b633922fbb96ffa15aa: start","job_status":"start","queue_duration_s":1.019874,"scheduling_latency_s":1.019916} gitlab # [ 149.446279] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.738Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3fe56b633922fbb96ffa15aa","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3fe56b633922fbb96ffa15aa: done: 0.024303 sec","job_status":"done","queue_duration_s":1.019874,"scheduling_latency_s":1.019916,"redis_calls":13,"redis_duration_s":0.003786,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001242,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002544,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015209,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024303,"completed_at":"2026-10-10T10:47:06.738Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.467876] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.740Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0d8cbb16baa42efbf85e8c7","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0d8cbb16baa42efbf85e8c7: start","job_status":"start","queue_duration_s":1.046112,"scheduling_latency_s":1.046141} gitlab # [ 149.469839] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.761Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0d8cbb16baa42efbf85e8c7","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0d8cbb16baa42efbf85e8c7: done: 0.021606 sec","job_status":"done","queue_duration_s":1.046112,"scheduling_latency_s":1.046141,"redis_calls":13,"redis_duration_s":0.003227,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001076,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002151,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012274,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021606,"completed_at":"2026-10-10T10:47:06.761Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.530513] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.822Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a7a69d2491e3149ac3eda8b2","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a7a69d2491e3149ac3eda8b2: start","job_status":"start","queue_duration_s":1.128173,"scheduling_latency_s":1.128222} gitlab # [ 149.563715] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.855Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a7a69d2491e3149ac3eda8b2","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a7a69d2491e3149ac3eda8b2: done: 0.033242 sec","job_status":"done","queue_duration_s":1.128173,"scheduling_latency_s":1.128222,"redis_calls":13,"redis_duration_s":0.0026939999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000935,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001759,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015977,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.033242,"completed_at":"2026-10-10T10:47:06.855Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.601392] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.893Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0eae188feb10ba863d4ded3c","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0eae188feb10ba863d4ded3c: start","job_status":"start","queue_duration_s":1.199209,"scheduling_latency_s":1.199253} gitlab # [ 149.625992] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.917Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0eae188feb10ba863d4ded3c","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0eae188feb10ba863d4ded3c: done: 0.024646 sec","job_status":"done","queue_duration_s":1.199209,"scheduling_latency_s":1.199253,"redis_calls":13,"redis_duration_s":0.003803,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001203,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0026,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024646,"completed_at":"2026-10-10T10:47:06.917Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.662504] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.954Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fccf947a4403b25098e1ed63","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fccf947a4403b25098e1ed63: start","job_status":"start","queue_duration_s":1.260337,"scheduling_latency_s":1.260376} gitlab # [ 149.697466] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:06.989Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fccf947a4403b25098e1ed63","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fccf947a4403b25098e1ed63: done: 0.034847 sec","job_status":"done","queue_duration_s":1.260337,"scheduling_latency_s":1.260376,"redis_calls":13,"redis_duration_s":0.002777,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000907,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00187,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015678,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.034847,"completed_at":"2026-10-10T10:47:06.989Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.740265] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:07.032Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0dea275b6dcf937ea23ebbf7","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0dea275b6dcf937ea23ebbf7: start","job_status":"start","queue_duration_s":1.338081,"scheduling_latency_s":1.338124} gitlab # [ 149.766724] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:07.058Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0dea275b6dcf937ea23ebbf7","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0dea275b6dcf937ea23ebbf7: done: 0.026415 sec","job_status":"done","queue_duration_s":1.338081,"scheduling_latency_s":1.338124,"redis_calls":13,"redis_duration_s":0.004005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002451,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015621,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026415,"completed_at":"2026-10-10T10:47:07.058Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.802817] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:07.094Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9bde59d7bbf0861bf6f679b","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9bde59d7bbf0861bf6f679b: start","job_status":"start","queue_duration_s":1.400639,"scheduling_latency_s":1.400678} gitlab # [ 149.828961] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:07.120Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9bde59d7bbf0861bf6f679b","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9bde59d7bbf0861bf6f679b: done: 0.025709 sec","job_status":"done","queue_duration_s":1.400639,"scheduling_latency_s":1.400678,"redis_calls":13,"redis_duration_s":0.004052,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001644,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002408,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015745,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025709,"completed_at":"2026-10-10T10:47:07.120Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.864302] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:07.156Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"005188fbfab39e5c1c2c2814","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-005188fbfab39e5c1c2c2814: start","job_status":"start","queue_duration_s":1.462351,"scheduling_latency_s":1.462375} gitlab # [ 149.900328] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:07.191Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"005188fbfab39e5c1c2c2814","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-005188fbfab39e5c1c2c2814: done: 0.035388 sec","job_status":"done","queue_duration_s":1.462351,"scheduling_latency_s":1.462375,"redis_calls":13,"redis_duration_s":0.00368,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001152,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002528,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01531,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035388,"completed_at":"2026-10-10T10:47:07.191Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.936820] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:07.228Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d38474602d6eb3e6490de58","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d38474602d6eb3e6490de58: start","job_status":"start","queue_duration_s":1.534703,"scheduling_latency_s":1.534743} gitlab # [ 149.973313] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:07.264Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d38474602d6eb3e6490de58","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d38474602d6eb3e6490de58: done: 0.035673 sec","job_status":"done","queue_duration_s":1.534703,"scheduling_latency_s":1.534743,"redis_calls":13,"redis_duration_s":0.003151,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000977,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002174,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017771,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035673,"completed_at":"2026-10-10T10:47:07.264Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.015806] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:07.307Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d4e77578eece7bd0b420c843","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d4e77578eece7bd0b420c843: start","job_status":"start","queue_duration_s":1.613843,"scheduling_latency_s":1.61387} gitlab # [ 150.054228] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:07.346Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d4e77578eece7bd0b420c843","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d4e77578eece7bd0b420c843: done: 0.038132 sec","job_status":"done","queue_duration_s":1.613843,"scheduling_latency_s":1.61387,"redis_calls":13,"redis_duration_s":0.003835,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001222,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002613,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015885,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038132,"completed_at":"2026-10-10T10:47:07.345Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.094166] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:07.385Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f0d13e55eab3e786b33278f","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f0d13e55eab3e786b33278f: start","job_status":"start","queue_duration_s":1.691838,"scheduling_latency_s":1.691879} gitlab # [ 150.127203] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:07.418Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f0d13e55eab3e786b33278f","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f0d13e55eab3e786b33278f: done: 0.033137 sec","job_status":"done","queue_duration_s":1.691838,"scheduling_latency_s":1.691879,"redis_calls":13,"redis_duration_s":0.00262,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001694,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016468,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.033137,"completed_at":"2026-10-10T10:47:07.418Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.164658] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:07.456Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c800b3624c64b9ae093653b2","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c800b3624c64b9ae093653b2: start","job_status":"start","queue_duration_s":1.762571,"scheduling_latency_s":1.76261} gitlab # [ 150.189293] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:07.480Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c800b3624c64b9ae093653b2","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c800b3624c64b9ae093653b2: done: 0.024211 sec","job_status":"done","queue_duration_s":1.762571,"scheduling_latency_s":1.76261,"redis_calls":13,"redis_duration_s":0.003738,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001266,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002472,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015559,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024211,"completed_at":"2026-10-10T10:47:07.480Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.226861] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:07.518Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e63bae72dcc8918cbe9a04e","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e63bae72dcc8918cbe9a04e: start","job_status":"start","queue_duration_s":1.824369,"scheduling_latency_s":1.824408} gitlab # [ 150.250952] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:07.542Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e63bae72dcc8918cbe9a04e","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e63bae72dcc8918cbe9a04e: done: 0.024441 sec","job_status":"done","queue_duration_s":1.824369,"scheduling_latency_s":1.824408,"redis_calls":13,"redis_duration_s":0.003916,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001227,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002689,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015545,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024441,"completed_at":"2026-10-10T10:47:07.542Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.287583] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:07.579Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83e4dc8a84849cbb52a4b186","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83e4dc8a84849cbb52a4b186: start","job_status":"start","queue_duration_s":1.885425,"scheduling_latency_s":1.88552} gitlab # [ 150.319556] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:07.611Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83e4dc8a84849cbb52a4b186","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83e4dc8a84849cbb52a4b186: done: 0.031823 sec","job_status":"done","queue_duration_s":1.885425,"scheduling_latency_s":1.88552,"redis_calls":13,"redis_duration_s":0.002518,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000886,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001632,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015434,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.031823,"completed_at":"2026-10-10T10:47:07.611Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.356605] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:07.648Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0809dc18079f50756e70d6ef","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0809dc18079f50756e70d6ef: start","job_status":"start","queue_duration_s":1.954346,"scheduling_latency_s":1.954387} gitlab # [ 150.379216] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:07.671Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0809dc18079f50756e70d6ef","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0809dc18079f50756e70d6ef: done: 0.023026 sec","job_status":"done","queue_duration_s":1.954346,"scheduling_latency_s":1.954387,"redis_calls":13,"redis_duration_s":0.004345,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001626,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002719,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014566,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023026,"completed_at":"2026-10-10T10:47:07.671Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.415487] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:07.707Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf865b5f6d4eb00c001c3c10","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf865b5f6d4eb00c001c3c10: start","job_status":"start","queue_duration_s":2.013607,"scheduling_latency_s":2.01363} gitlab # [ 150.451899] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:07.743Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf865b5f6d4eb00c001c3c10","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf865b5f6d4eb00c001c3c10: done: 0.03541 sec","job_status":"done","queue_duration_s":2.013607,"scheduling_latency_s":2.01363,"redis_calls":13,"redis_duration_s":0.003606,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001153,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002453,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014761,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.03541,"completed_at":"2026-10-10T10:47:07.743Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.487892] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:07.779Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"06e8cbc5ce2aa06f7ad4cd80","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-06e8cbc5ce2aa06f7ad4cd80: start","job_status":"start","queue_duration_s":2.085808,"scheduling_latency_s":2.085849} gitlab # [ 150.520401] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:07.812Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"06e8cbc5ce2aa06f7ad4cd80","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-06e8cbc5ce2aa06f7ad4cd80: done: 0.032395 sec","job_status":"done","queue_duration_s":2.085808,"scheduling_latency_s":2.085849,"redis_calls":13,"redis_duration_s":0.0035970000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001152,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002445,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01464,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032395,"completed_at":"2026-10-10T10:47:07.812Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.557218] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:07.849Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c64b1539875bdd1fe83a56b9","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c64b1539875bdd1fe83a56b9: start","job_status":"start","queue_duration_s":2.155086,"scheduling_latency_s":2.155124} gitlab # [ 150.589342] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:07.880Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c64b1539875bdd1fe83a56b9","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c64b1539875bdd1fe83a56b9: done: 0.03172 sec","job_status":"done","queue_duration_s":2.155086,"scheduling_latency_s":2.155124,"redis_calls":13,"redis_duration_s":0.0023959999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000763,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001633,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015303,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.03172,"completed_at":"2026-10-10T10:47:07.880Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.626820] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:07.918Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b422fa8b792d97e4d088e239","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b422fa8b792d97e4d088e239: start","job_status":"start","queue_duration_s":2.224629,"scheduling_latency_s":2.224669} gitlab # [ 150.660654] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:07.951Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b422fa8b792d97e4d088e239","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b422fa8b792d97e4d088e239: done: 0.033322 sec","job_status":"done","queue_duration_s":2.224629,"scheduling_latency_s":2.224669,"redis_calls":13,"redis_duration_s":0.002344,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000759,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001585,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015415,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.033322,"completed_at":"2026-10-10T10:47:07.951Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.698689] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:07.990Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc26e243181dc809a1a95c51","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc26e243181dc809a1a95c51: start","job_status":"start","queue_duration_s":2.296574,"scheduling_latency_s":2.296615} gitlab # [ 150.732568] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:08.024Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc26e243181dc809a1a95c51","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc26e243181dc809a1a95c51: done: 0.033503 sec","job_status":"done","queue_duration_s":2.296574,"scheduling_latency_s":2.296615,"redis_calls":13,"redis_duration_s":0.002534,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000894,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00164,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016573,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.033503,"completed_at":"2026-10-10T10:47:08.023Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.770355] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:08.062Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81345d6f5bbb2341a5d97300","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81345d6f5bbb2341a5d97300: start","job_status":"start","queue_duration_s":2.368184,"scheduling_latency_s":2.368226} gitlab # [ 150.803103] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:08.094Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81345d6f5bbb2341a5d97300","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81345d6f5bbb2341a5d97300: done: 0.032688 sec","job_status":"done","queue_duration_s":2.368184,"scheduling_latency_s":2.368226,"redis_calls":13,"redis_duration_s":0.002652,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001735,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015676,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032688,"completed_at":"2026-10-10T10:47:08.094Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.840884] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:08.132Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5d21b486f5e3c573ffa43ad","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5d21b486f5e3c573ffa43ad: start","job_status":"start","queue_duration_s":2.438812,"scheduling_latency_s":2.438853} gitlab # [ 150.875648] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:08.167Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5d21b486f5e3c573ffa43ad","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5d21b486f5e3c573ffa43ad: done: 0.034563 sec","job_status":"done","queue_duration_s":2.438812,"scheduling_latency_s":2.438853,"redis_calls":13,"redis_duration_s":0.003592,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001105,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002487,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016605,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.034563,"completed_at":"2026-10-10T10:47:08.167Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.916464] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:08.208Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a5226dcfbac4c719212830d","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a5226dcfbac4c719212830d: start","job_status":"start","queue_duration_s":2.514297,"scheduling_latency_s":2.514341} gitlab # [ 150.949327] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:08.240Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a5226dcfbac4c719212830d","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a5226dcfbac4c719212830d: done: 0.032617 sec","job_status":"done","queue_duration_s":2.514297,"scheduling_latency_s":2.514341,"redis_calls":13,"redis_duration_s":0.0036580000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000942,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002716,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032617,"completed_at":"2026-10-10T10:47:08.240Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.986351] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:08.278Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8435ed60e31eed951ed549dc","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8435ed60e31eed951ed549dc: start","job_status":"start","queue_duration_s":2.584177,"scheduling_latency_s":2.584221} gitlab # [ 151.010907] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:08.302Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8435ed60e31eed951ed549dc","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8435ed60e31eed951ed549dc: done: 0.024595 sec","job_status":"done","queue_duration_s":2.584177,"scheduling_latency_s":2.584221,"redis_calls":13,"redis_duration_s":0.003464,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001113,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002351,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015745,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024595,"completed_at":"2026-10-10T10:47:08.302Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.048570] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:08.340Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff2eece5a5402e48e499d28a","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff2eece5a5402e48e499d28a: start","job_status":"start","queue_duration_s":2.646327,"scheduling_latency_s":2.646367} gitlab # [ 151.081811] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:08.373Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff2eece5a5402e48e499d28a","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff2eece5a5402e48e499d28a: done: 0.03288 sec","job_status":"done","queue_duration_s":2.646327,"scheduling_latency_s":2.646367,"redis_calls":13,"redis_duration_s":0.002588,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000949,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001639,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016301,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.03288,"completed_at":"2026-10-10T10:47:08.373Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.119106] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:08.410Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ee21647899d6f0c45776a6b","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ee21647899d6f0c45776a6b: start","job_status":"start","queue_duration_s":2.716806,"scheduling_latency_s":2.716846} gitlab # [ 151.150850] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:08.442Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ee21647899d6f0c45776a6b","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ee21647899d6f0c45776a6b: done: 0.031645 sec","job_status":"done","queue_duration_s":2.716806,"scheduling_latency_s":2.716846,"redis_calls":13,"redis_duration_s":0.002481,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000826,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001655,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014677,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.031645,"completed_at":"2026-10-10T10:47:08.442Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.188638] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:08.480Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"155ca99f850c04624fcb59fc","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-155ca99f850c04624fcb59fc: start","job_status":"start","queue_duration_s":2.786305,"scheduling_latency_s":2.786347} gitlab # [ 151.268887] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:08.560Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"155ca99f850c04624fcb59fc","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-155ca99f850c04624fcb59fc: done: 0.080642 sec","job_status":"done","queue_duration_s":2.786305,"scheduling_latency_s":2.786347,"redis_calls":13,"redis_duration_s":0.003811,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001063,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002748,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.069853,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.080642,"completed_at":"2026-10-10T10:47:08.560Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.332594] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:08.624Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6fec8498b64903cd6aad5a57","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6fec8498b64903cd6aad5a57: start","job_status":"start","queue_duration_s":2.930297,"scheduling_latency_s":2.930346} gitlab # [ 151.357624] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:08.649Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6fec8498b64903cd6aad5a57","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6fec8498b64903cd6aad5a57: done: 0.025158 sec","job_status":"done","queue_duration_s":2.930297,"scheduling_latency_s":2.930346,"redis_calls":13,"redis_duration_s":0.004018,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00121,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002808,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015363,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025158,"completed_at":"2026-10-10T10:47:08.649Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.394427] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:08.686Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"13f21dce485148801cea597c","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-13f21dce485148801cea597c: start","job_status":"start","queue_duration_s":2.992189,"scheduling_latency_s":2.992229} gitlab # [ 151.429179] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:08.720Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"13f21dce485148801cea597c","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-13f21dce485148801cea597c: done: 0.03408 sec","job_status":"done","queue_duration_s":2.992189,"scheduling_latency_s":2.992229,"redis_calls":13,"redis_duration_s":0.005153,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001614,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003539,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014507,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.03408,"completed_at":"2026-10-10T10:47:08.720Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.467410] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:08.759Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c98dd802eaf33e1e476e24a0","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c98dd802eaf33e1e476e24a0: start","job_status":"start","queue_duration_s":3.064879,"scheduling_latency_s":3.064927} gitlab # [ 151.500064] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:08.791Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c98dd802eaf33e1e476e24a0","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c98dd802eaf33e1e476e24a0: done: 0.032928 sec","job_status":"done","queue_duration_s":3.064879,"scheduling_latency_s":3.064927,"redis_calls":13,"redis_duration_s":0.002415,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000783,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001632,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014908,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032928,"completed_at":"2026-10-10T10:47:08.791Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.536482] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:08.828Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"01cbca9777a3f261ae1915bc","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-01cbca9777a3f261ae1915bc: start","job_status":"start","queue_duration_s":3.13429,"scheduling_latency_s":3.134332} gitlab # [ 151.567948] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:08.859Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"01cbca9777a3f261ae1915bc","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-01cbca9777a3f261ae1915bc: done: 0.031423 sec","job_status":"done","queue_duration_s":3.13429,"scheduling_latency_s":3.134332,"redis_calls":13,"redis_duration_s":0.00236,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000847,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001513,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.031423,"completed_at":"2026-10-10T10:47:08.859Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.604225] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:08.896Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0cb262afaabc347080adaa45","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0cb262afaabc347080adaa45: start","job_status":"start","queue_duration_s":3.202008,"scheduling_latency_s":3.20205} gitlab # [ 151.635383] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:08.927Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0cb262afaabc347080adaa45","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0cb262afaabc347080adaa45: done: 0.031077 sec","job_status":"done","queue_duration_s":3.202008,"scheduling_latency_s":3.20205,"redis_calls":13,"redis_duration_s":0.002264,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000754,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00151,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014816,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.031077,"completed_at":"2026-10-10T10:47:08.927Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.672480] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:08.963Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9871d75dea72d03c970bd91f","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9871d75dea72d03c970bd91f: start","job_status":"start","queue_duration_s":3.269729,"scheduling_latency_s":3.269769} gitlab # [ 151.701600] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:08.992Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9871d75dea72d03c970bd91f","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9871d75dea72d03c970bd91f: done: 0.029261 sec","job_status":"done","queue_duration_s":3.269729,"scheduling_latency_s":3.269769,"redis_calls":13,"redis_duration_s":0.002742,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001072,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00167,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012739,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029261,"completed_at":"2026-10-10T10:47:08.992Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.741080] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.032Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e800c27b4817c50f3b7de216","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e800c27b4817c50f3b7de216: start","job_status":"start","queue_duration_s":3.338714,"scheduling_latency_s":3.338752} gitlab # [ 151.765315] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.057Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e800c27b4817c50f3b7de216","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e800c27b4817c50f3b7de216: done: 0.024405 sec","job_status":"done","queue_duration_s":3.338714,"scheduling_latency_s":3.338752,"redis_calls":13,"redis_duration_s":0.003928,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001306,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002622,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015215,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024405,"completed_at":"2026-10-10T10:47:09.057Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.802908] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.094Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bddbd19d8b2f3855c19e196b","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bddbd19d8b2f3855c19e196b: start","job_status":"start","queue_duration_s":3.400676,"scheduling_latency_s":3.400723} gitlab # [ 151.827460] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.119Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bddbd19d8b2f3855c19e196b","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bddbd19d8b2f3855c19e196b: done: 0.024625 sec","job_status":"done","queue_duration_s":3.400676,"scheduling_latency_s":3.400723,"redis_calls":13,"redis_duration_s":0.0036889999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001181,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002508,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015231,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024625,"completed_at":"2026-10-10T10:47:09.119Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.865509] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.157Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b933e9fd27f074ce3f7f1786","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b933e9fd27f074ce3f7f1786: start","job_status":"start","queue_duration_s":3.463254,"scheduling_latency_s":3.463301} gitlab # [ 151.889619] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.181Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b933e9fd27f074ce3f7f1786","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b933e9fd27f074ce3f7f1786: done: 0.024183 sec","job_status":"done","queue_duration_s":3.463254,"scheduling_latency_s":3.463301,"redis_calls":13,"redis_duration_s":0.0036550000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001182,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002473,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014842,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024183,"completed_at":"2026-10-10T10:47:09.181Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.926548] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.218Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aba7a5fb0e3ec2a590a039f6","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aba7a5fb0e3ec2a590a039f6: start","job_status":"start","queue_duration_s":3.524275,"scheduling_latency_s":3.524313} gitlab # [ 151.950277] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.241Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aba7a5fb0e3ec2a590a039f6","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aba7a5fb0e3ec2a590a039f6: done: 0.023212 sec","job_status":"done","queue_duration_s":3.524275,"scheduling_latency_s":3.524313,"redis_calls":13,"redis_duration_s":0.0037280000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001201,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002527,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013939,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023212,"completed_at":"2026-10-10T10:47:09.241Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.986693] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.278Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98d168667c982cd4b22f9ab3","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98d168667c982cd4b22f9ab3: start","job_status":"start","queue_duration_s":3.584495,"scheduling_latency_s":3.584533} gitlab # [ 152.009784] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.301Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98d168667c982cd4b22f9ab3","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98d168667c982cd4b22f9ab3: done: 0.023032 sec","job_status":"done","queue_duration_s":3.584495,"scheduling_latency_s":3.584533,"redis_calls":13,"redis_duration_s":0.0036360000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001184,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002452,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01463,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023032,"completed_at":"2026-10-10T10:47:09.301Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.047129] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.339Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b5ad64da3469e5667a73f24e","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b5ad64da3469e5667a73f24e: start","job_status":"start","queue_duration_s":3.644991,"scheduling_latency_s":3.645031} gitlab # [ 152.077885] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.369Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b5ad64da3469e5667a73f24e","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b5ad64da3469e5667a73f24e: done: 0.03066 sec","job_status":"done","queue_duration_s":3.644991,"scheduling_latency_s":3.645031,"redis_calls":13,"redis_duration_s":0.002265,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000759,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001506,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014572,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.03066,"completed_at":"2026-10-10T10:47:09.369Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.114973] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.407Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b5fbaee95b3b5d1d40e01919","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b5fbaee95b3b5d1d40e01919: start","job_status":"start","queue_duration_s":3.712868,"scheduling_latency_s":3.712909} gitlab # [ 152.139545] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.431Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b5fbaee95b3b5d1d40e01919","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b5fbaee95b3b5d1d40e01919: done: 0.024408 sec","job_status":"done","queue_duration_s":3.712868,"scheduling_latency_s":3.712909,"redis_calls":13,"redis_duration_s":0.00413,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001367,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002763,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01511,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024408,"completed_at":"2026-10-10T10:47:09.431Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.179127] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.470Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62178ca64a410eb23d16b0c0","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62178ca64a410eb23d16b0c0: start","job_status":"start","queue_duration_s":3.776824,"scheduling_latency_s":3.776873} gitlab # [ 152.203565] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.495Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62178ca64a410eb23d16b0c0","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62178ca64a410eb23d16b0c0: done: 0.024565 sec","job_status":"done","queue_duration_s":3.776824,"scheduling_latency_s":3.776873,"redis_calls":13,"redis_duration_s":0.003755,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001242,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002513,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01449,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024565,"completed_at":"2026-10-10T10:47:09.495Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.240422] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.532Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"298249cdc70a19f8af427c83","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-298249cdc70a19f8af427c83: start","job_status":"start","queue_duration_s":3.838164,"scheduling_latency_s":3.838204} gitlab # [ 152.271720] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.563Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"298249cdc70a19f8af427c83","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-298249cdc70a19f8af427c83: done: 0.031116 sec","job_status":"done","queue_duration_s":3.838164,"scheduling_latency_s":3.838204,"redis_calls":13,"redis_duration_s":0.002589,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000911,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001678,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014621,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.031116,"completed_at":"2026-10-10T10:47:09.563Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.309141] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.601Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d11df390fd29cb2b4f20dfc","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d11df390fd29cb2b4f20dfc: start","job_status":"start","queue_duration_s":3.906888,"scheduling_latency_s":3.906937} gitlab # [ 152.333661] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.625Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d11df390fd29cb2b4f20dfc","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d11df390fd29cb2b4f20dfc: done: 0.024601 sec","job_status":"done","queue_duration_s":3.906888,"scheduling_latency_s":3.906937,"redis_calls":13,"redis_duration_s":0.003678,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001185,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002493,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015185,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024601,"completed_at":"2026-10-10T10:47:09.625Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.370827] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.662Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99da73e2ff27b8629712e1de","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99da73e2ff27b8629712e1de: start","job_status":"start","queue_duration_s":3.968563,"scheduling_latency_s":3.968613} gitlab # [ 152.402991] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.694Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99da73e2ff27b8629712e1de","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99da73e2ff27b8629712e1de: done: 0.03223 sec","job_status":"done","queue_duration_s":3.968563,"scheduling_latency_s":3.968613,"redis_calls":13,"redis_duration_s":0.0027140000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000845,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001869,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015139,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.03223,"completed_at":"2026-10-10T10:47:09.694Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.440331] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.732Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9875b75c7ef401bbcaefd54c","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9875b75c7ef401bbcaefd54c: start","job_status":"start","queue_duration_s":4.038126,"scheduling_latency_s":4.038168} gitlab # [ 152.472666] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.763Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9875b75c7ef401bbcaefd54c","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9875b75c7ef401bbcaefd54c: done: 0.030872 sec","job_status":"done","queue_duration_s":4.038126,"scheduling_latency_s":4.038168,"redis_calls":13,"redis_duration_s":0.002518,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000875,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001643,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014162,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.030872,"completed_at":"2026-10-10T10:47:09.762Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.475196] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.766Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47177db08d15359758d2191e","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47177db08d15359758d2191e: start","job_status":"start","queue_duration_s":4.072019,"scheduling_latency_s":4.072049} gitlab # [ 152.490594] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.782Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47177db08d15359758d2191e","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47177db08d15359758d2191e: done: 0.016754 sec","job_status":"done","queue_duration_s":4.072019,"scheduling_latency_s":4.072049,"redis_calls":13,"redis_duration_s":0.0027660000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000848,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001918,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01027,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016754,"completed_at":"2026-10-10T10:47:09.782Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.516861] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.783Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cff2be650a60b1786aca9ac7","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cff2be650a60b1786aca9ac7: start","job_status":"start","queue_duration_s":4.089602,"scheduling_latency_s":4.089615} gitlab # [ 152.528194] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.792Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cff2be650a60b1786aca9ac7","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cff2be650a60b1786aca9ac7: done: 0.009169 sec","job_status":"done","queue_duration_s":4.089602,"scheduling_latency_s":4.089615,"redis_calls":13,"redis_duration_s":0.001494,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000985,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005898,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009169,"completed_at":"2026-10-10T10:47:09.792Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.595517] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.793Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d1e8fe39e3560fc2d3e7ed2","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d1e8fe39e3560fc2d3e7ed2: start","job_status":"start","queue_duration_s":4.099576,"scheduling_latency_s":4.099591} gitlab # [ 152.596304] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.802Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d1e8fe39e3560fc2d3e7ed2","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d1e8fe39e3560fc2d3e7ed2: done: 0.009103 sec","job_status":"done","queue_duration_s":4.099576,"scheduling_latency_s":4.099591,"redis_calls":13,"redis_duration_s":0.001501,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000996,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005954,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009103,"completed_at":"2026-10-10T10:47:09.802Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.608602] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.803Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48a712c645ae179bb4d13fa4","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48a712c645ae179bb4d13fa4: start","job_status":"start","queue_duration_s":4.10949,"scheduling_latency_s":4.109503} gitlab # [ 152.619696] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.812Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48a712c645ae179bb4d13fa4","created_at":"2026-10-10T10:47:05.627Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-6caaf82f52af47fe","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6b7feb3dffdc639440053b75347b26da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:05.694Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48a712c645ae179bb4d13fa4: done: 0.00914 sec","job_status":"done","queue_duration_s":4.10949,"scheduling_latency_s":4.109503,"redis_calls":13,"redis_duration_s":0.0014789999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000478,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001001,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005456,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00914,"completed_at":"2026-10-10T10:47:09.812Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.670075] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.814Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9098eade38266b001eb3bac9","created_at":"2026-10-10T10:47:06.352Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-c7d85e8fc9724a05","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3353226adb1f3b5534340e9a9c47a1e6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:06.399Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9098eade38266b001eb3bac9: start","job_status":"start","queue_duration_s":3.414665,"scheduling_latency_s":3.414679} gitlab # [ 152.670836] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.827Z","message":"No existing merge request to be cleaned up."} gitlab # [ 152.671565] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.836Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9098eade38266b001eb3bac9","created_at":"2026-10-10T10:47:06.352Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-c7d85e8fc9724a05","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3353226adb1f3b5534340e9a9c47a1e6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:06.399Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9098eade38266b001eb3bac9: done: 0.022181 sec","job_status":"done","queue_duration_s":3.414665,"scheduling_latency_s":3.414679,"redis_calls":13,"redis_duration_s":0.001689,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000277,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000638,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000774,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011942,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000219301,"duration_s":0.022181,"completed_at":"2026-10-10T10:47:09.836Z","load_balancing_strategy":"primary","db_duration_s":0.005668,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.723283] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.837Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ae235bbf2dbdafddd1a73c91","created_at":"2026-10-10T10:47:06.352Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-c7d85e8fc9724a05","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3353226adb1f3b5534340e9a9c47a1e6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:06.399Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ae235bbf2dbdafddd1a73c91: start","job_status":"start","queue_duration_s":3.437722,"scheduling_latency_s":3.437735} gitlab # [ 152.723892] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.844Z","message":"No existing merge request to be cleaned up."} gitlab # [ 152.724578] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.849Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ae235bbf2dbdafddd1a73c91","created_at":"2026-10-10T10:47:06.352Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-c7d85e8fc9724a05","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3353226adb1f3b5534340e9a9c47a1e6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:06.399Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ae235bbf2dbdafddd1a73c91: done: 0.011668 sec","job_status":"done","queue_duration_s":3.437722,"scheduling_latency_s":3.437735,"redis_calls":10,"redis_duration_s":0.001251,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000713,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007638,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011668,"completed_at":"2026-10-10T10:47:09.849Z","load_balancing_strategy":"primary","db_duration_s":0.00166,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.765954] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.849Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"097aff49b127ef477feb0499","created_at":"2026-10-10T10:47:06.352Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-c7d85e8fc9724a05","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3353226adb1f3b5534340e9a9c47a1e6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:06.399Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-097aff49b127ef477feb0499: start","job_status":"start","queue_duration_s":3.450205,"scheduling_latency_s":3.450218} gitlab # [ 152.766664] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.856Z","message":"No existing merge request to be cleaned up."} gitlab # [ 152.767199] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.861Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"097aff49b127ef477feb0499","created_at":"2026-10-10T10:47:06.352Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-c7d85e8fc9724a05","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3353226adb1f3b5534340e9a9c47a1e6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:06.399Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-097aff49b127ef477feb0499: done: 0.011272 sec","job_status":"done","queue_duration_s":3.450205,"scheduling_latency_s":3.450218,"redis_calls":10,"redis_duration_s":0.0012829999999999999,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000762,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007339,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011272,"completed_at":"2026-10-10T10:47:09.861Z","load_balancing_strategy":"primary","db_duration_s":0.000892,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.821761] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.861Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ca96595aaed32f0ac6dd611d","created_at":"2026-10-10T10:47:06.352Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-c7d85e8fc9724a05","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3353226adb1f3b5534340e9a9c47a1e6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:06.399Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ca96595aaed32f0ac6dd611d: start","job_status":"start","queue_duration_s":3.462287,"scheduling_latency_s":3.4623} gitlab # [ 152.822591] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.868Z","message":"No existing merge request to be cleaned up."} gitlab # [ 152.823134] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:09.872Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ca96595aaed32f0ac6dd611d","created_at":"2026-10-10T10:47:06.352Z","trace_propagation_headers":{"sentry-trace":"47a246416cb146238d3cba1895d1045a-c7d85e8fc9724a05","baggage":"sentry-trace_id=47a246416cb146238d3cba1895d1045a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3353226adb1f3b5534340e9a9c47a1e6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:06.399Z","job_size_bytes":2,"pid":1617,"sidekiq_tid":"maex","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ca96595aaed32f0ac6dd611d: done: 0.010676 sec","job_status":"done","queue_duration_s":3.462287,"scheduling_latency_s":3.4623,"redis_calls":10,"redis_duration_s":0.001299,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000763,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007201,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010676,"completed_at":"2026-10-10T10:47:09.872Z","load_balancing_strategy":"primary","db_duration_s":0.000945,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 161.745267] setup-gitlab-start[2227]: W, [2026-10-10T10:47:19.036888 #2227] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 183.141747] postgres[2431]: [2431] ERROR: relation "table_without_models" does not exist gitlab # [ 183.143001] postgres[2431]: [2431] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 186.105222] runuser[2226]: pam_unix(runuser:session): session closed for user gitlab gitlab # [ 186.106711] setup-gitlab-start[2204]: + systemctl restart gitlab-sidekiq.service gitlab # [ 186.132315] systemd[1]: Stopping gitlab-sidekiq.service... gitlab # [ 186.133502] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:43.424Z","message":"Shutting down"} gitlab # [ 186.134859] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:43.424Z","message":"Scheduler exiting..."} gitlab # [ 186.136293] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:43.425Z","message":"Terminating quiet threads for default capsule"} gitlab # [ 186.137930] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:43.425Z","message":"Scheduler exiting..."} gitlab # [ 186.634553] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:43.926Z","message":"Pausing to allow jobs to finish..."} gitlab # [ 188.138192] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:45.429Z","message":"Bye!"} gitlab # [ 188.163089] sidekiq-cluster[1617]: {"severity":"INFO","time":"2026-10-10T10:47:45.455Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1617,"worker_id":"sidekiq_0","memwd_rss_bytes":935759872,"retry":0} gitlab # [ 191.145398] systemd[1]: gitlab-sidekiq.service: Deactivated successfully. gitlab # [ 191.147111] systemd[1]: Stopped gitlab-sidekiq.service. gitlab # [ 191.148724] systemd[1]: gitlab-sidekiq.service: Consumed 56.876s CPU time over 1min 58.177s wall clock time, 955.1M memory peak, 4.4M read from disk. gitlab # [ 191.150867] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 191.155736] setup-gitlab-start[2204]: + mkdir -p /var/lib/secrets gitlab # [ 191.158937] setup-gitlab-start[2204]: + echo nixbot-test-token-of-20-chars gitlab # [ 191.160482] setup-gitlab-start[2204]: + chmod 644 /var/lib/secrets/gitlab-token gitlab # [ 191.166704] setup-gitlab-start[2204]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars gitlab # [ 191.632824] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:47:48 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 426 gitlab # [ 192.099538] gitaly[893]: time="2026-10-10T10:47:49.391Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT5YFX9KR5AKV1T0D8K9T error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T10:47:59.010" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:49.390" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.101351] gitaly[893]: time="2026-10-10T10:47:49.391Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT5YFX9KR5AKV1T0D8K9T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T10:47:59.010" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:49.390" grpc.time_ms=0.252 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.262714] gitaly[893]: time="2026-10-10T10:47:49.554Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT5YFX9KR5AKV1T0D8K9T error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T10:47:59.174" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:49.554" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.264477] gitaly[893]: time="2026-10-10T10:47:49.555Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT5YFX9KR5AKV1T0D8K9T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T10:47:59.174" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:49.554" grpc.time_ms=0.391 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.377755] gitaly[893]: time="2026-10-10T10:47:49.669Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT5YFX9KR5AKV1T0D8K9T diskcache=23e49b8a-6c32-4794-a3af-380bf74c7038 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-10T10:47:59.229" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:49.609" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.379473] gitaly[893]: time="2026-10-10T10:47:49.670Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=21 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1568 command.oublock=32 command.real_time_ms=25 command.spawn_token_fork_ms=1 command.system_time_ms=17 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT5YFX9KR5AKV1T0D8K9T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-10T10:47:59.229" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:49.609" grpc.time_ms=60.14 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.442338] gitaly[893]: time="2026-10-10T10:47:49.734Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT5YFX9KR5AKV1T0D8K9T grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T10:47:59.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:49.734" pid=893 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 192.444180] gitaly[893]: time="2026-10-10T10:47:49.734Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT5YFX9KR5AKV1T0D8K9T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T10:47:59.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:49.734" grpc.time_ms=0.735 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.484371] gitaly[893]: time="2026-10-10T10:47:49.776Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=620 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT5YFX9KR5AKV1T0D8K9T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-10T10:47:59.038" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:49.768" grpc.time_ms=8.092 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.561041] gitaly[893]: time="2026-10-10T10:47:49.853Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT5YFX9KR5AKV1T0D8K9T error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T10:47:59.122" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:49.853" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.561976] gitaly[893]: time="2026-10-10T10:47:49.853Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT5YFX9KR5AKV1T0D8K9T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T10:47:59.122" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:49.853" grpc.time_ms=0.388 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.644349] gitaly[893]: time="2026-10-10T10:47:49.936Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT5YFX9KR5AKV1T0D8K9T diskcache=77ea7210-1557-49c2-b6dd-edd1a3589257 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-10T10:47:59.126" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:49.896" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.645985] gitaly[893]: time="2026-10-10T10:47:49.936Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1585 command.oublock=32 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT5YFX9KR5AKV1T0D8K9T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-10T10:47:59.126" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:49.896" grpc.time_ms=40.472 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.687428] gitaly[893]: time="2026-10-10T10:47:49.979Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT5YFX9KR5AKV1T0D8K9T grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T10:47:59.248" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:49.978" pid=893 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 192.688819] gitaly[893]: time="2026-10-10T10:47:49.980Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT5YFX9KR5AKV1T0D8K9T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T10:47:59.248" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:49.978" grpc.time_ms=0.939 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.701239] gitaly[893]: time="2026-10-10T10:47:49.993Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT5YFX9KR5AKV1T0D8K9T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-10T10:47:59.253" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:49.983" grpc.time_ms=9.401 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.734853] gitaly[893]: time="2026-10-10T10:47:50.027Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT5YFX9KR5AKV1T0D8K9T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T10:48:00.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:50.026" grpc.time_ms=0.447 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.745518] gitaly[893]: time="2026-10-10T10:47:50.037Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT5YFX9KR5AKV1T0D8K9T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-10T10:48:00.016" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:50.037" grpc.time_ms=0.346 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 193.189359] gitaly[893]: time="2026-10-10T10:47:50.481Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPT5YFX9KR5AKV1T0D8K9T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-10T10:48:00.095" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:50.475" grpc.time_ms=5.725 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 193.566771] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:47:50 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 1914 gitlab # [ 193.571382] setup-gitlab-start[2490]: {"id":1,"message":null,"error":null} gitlab # [ 194.261512] sidekiq-cluster[2478]: {"severity":"INFO","time":"2026-10-10T10:47:51.536Z","message":"Starting cluster with 1 processes"} gitlab # [ 198.990493] gitaly[893]: time="2026-10-10T10:47:56.282Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTCQJW1FBYM81QZE2F415 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-10T10:48:06.001" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:56.272" grpc.time_ms=10.154 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 199.451168] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:47:56 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 849 gitlab # [ 199.457941] setup-gitlab-start[2204]: + mkdir -p /tmp/test-flake gitlab # [ 199.460105] setup-gitlab-start[2204]: + cd /tmp/test-flake gitlab # [ 199.461489] setup-gitlab-start[2204]: + git init -b master gitlab # [ 199.494070] setup-gitlab-start[2576]: Initialized empty Git repository in /tmp/test-flake/.git/ gitlab # [ 199.496851] setup-gitlab-start[2204]: + git config user.name test gitlab # [ 199.503271] setup-gitlab-start[2204]: + git config user.email test@example.com gitlab # [ 199.510624] setup-gitlab-start[2204]: + cat gitlab # [ 199.515277] setup-gitlab-start[2204]: + git add flake.nix gitlab # [ 199.522806] setup-gitlab-start[2204]: + git commit -m 'initial commit' gitlab # [ 199.552341] setup-gitlab-start[2581]: [master (root-commit) bb32878] initial commit gitlab # [ 199.554435] setup-gitlab-start[2581]: 1 file changed, 10 insertions(+) gitlab # [ 199.555549] setup-gitlab-start[2581]: create mode 100644 flake.nix gitlab # [ 199.557338] setup-gitlab-start[2204]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git gitlab # [ 199.569416] setup-gitlab-start[2204]: + git push -u origin master gitlab # [ 199.643634] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:47:56 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 39 gitlab # [ 199.745347] gitaly[893]: time="2026-10-10T10:47:57.037Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTDRD1TMVC3BDM8TPDJKP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:47:57.027" grpc.time_ms=10.251 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 199.748609] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:47:57 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 98 gitlab # [ 200.021262] gitaly[893]: time="2026-10-10T10:47:57.312Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-10T10:48:07.281" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:57.302" grpc.time_ms=10.28 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 200.099942] gitaly[893]: time="2026-10-10T10:47:57.392Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=332 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-10-10T10:48:27.079" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-10-10T10:47:57.380" grpc.time_ms=11.885 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 200.154893] gitaly[893]: time="2026-10-10T10:47:57.447Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 duration_ms=203ns method=POST pid=893 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 200.217923] gitaly[893]: time="2026-10-10T10:47:57.509Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 duration_ms=54ns method=POST pid=893 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 200.218745] gitaly[893]: time="2026-10-10T10:47:57.510Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5309 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:47:57.242" grpc.time_ms=267.389 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 200.280837] gitaly[893]: time="2026-10-10T10:47:57.573Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4779 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:47:57.572" grpc.time_ms=0.762 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 200.338911] gitaly[893]: time="2026-10-10T10:47:57.631Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4778 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:47:57.629" grpc.time_ms=1.588 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 200.387794] gitaly[893]: time="2026-10-10T10:47:57.680Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4780 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:47:57.679" grpc.time_ms=0.843 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 200.511224] gitaly[893]: time="2026-10-10T10:47:57.803Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-10T10:48:07.002" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:47:57.792" grpc.time_ms=10.755 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 200.572963] gitaly[893]: time="2026-10-10T10:47:57.865Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1558 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-10-10T10:48:07.048" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:47:57.838" grpc.time_ms=26.097 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 200.616090] gitaly[893]: time="2026-10-10T10:47:57.908Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 duration_ms=188ns method=POST pid=893 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 200.617225] gitaly[893]: time="2026-10-10T10:47:57.909Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4803 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:47:57.719" grpc.time_ms=189.712 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 200.652518] gitaly[893]: time="2026-10-10T10:47:57.944Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 diskcache=d6b96f20-15fe-46d1-a713-82aa89a3aee0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:47:57.140" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 200.653889] gitaly[893]: time="2026-10-10T10:47:57.945Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=260 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=14044 command.oublock=32 command.real_time_ms=792 command.spawn_token_fork_ms=1 command.system_time_ms=155 command.user_time_ms=105 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:47:57.140" grpc.time_ms=804.6 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 200.655498] gitlab-workhorse[827]: time="2026-10-10T10:47:57Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 200.656302] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:47:57 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 862 gitlab # [ 200.679452] setup-gitlab-start[2585]: To http://gitlab/root/test-flake.git gitlab # [ 200.680060] setup-gitlab-start[2585]: * [new branch] master -> master gitlab # [ 200.685179] setup-gitlab-start[2585]: branch 'master' set up to track 'origin/master'. gitlab # [ 200.690039] systemd[1]: Finished setup-gitlab.service. gitlab # [ 200.696305] systemd[1]: Starting nixbot CI service... gitlab: (finished: waiting for unit setup-gitlab.service, in 201.91 seconds) gitlab: waiting for unit nixbot.service gitlab # [ 202.052802] nixbot[2661]: {"ts": "2026-10-10T10:47:59.344Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} gitlab # [ 202.086984] nixbot[2661]: {"ts": "2026-10-10T10:47:59.379Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} gitlab # [ 202.114258] nixbot[2661]: {"ts": "2026-10-10T10:47:59.406Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} gitlab # [ 202.119919] nixbot[2661]: {"ts": "2026-10-10T10:47:59.412Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} gitlab # [ 202.125072] nixbot[2661]: {"ts": "2026-10-10T10:47:59.417Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} gitlab # [ 202.133148] nixbot[2661]: {"ts": "2026-10-10T10:47:59.425Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} gitlab # [ 202.142474] nixbot[2661]: {"ts": "2026-10-10T10:47:59.434Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} gitlab # [ 202.147455] nixbot[2661]: {"ts": "2026-10-10T10:47:59.437Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} gitlab # [ 202.152952] nixbot[2661]: {"ts": "2026-10-10T10:47:59.445Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} gitlab # [ 202.158120] nixbot[2661]: {"ts": "2026-10-10T10:47:59.448Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} gitlab # [ 202.162789] nixbot[2661]: {"ts": "2026-10-10T10:47:59.454Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} gitlab # [ 202.170945] nixbot[2661]: {"ts": "2026-10-10T10:47:59.463Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} gitlab # [ 202.176156] nixbot[2661]: {"ts": "2026-10-10T10:47:59.467Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} gitlab # [ 202.178429] nixbot[2661]: {"ts": "2026-10-10T10:47:59.470Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} gitlab # [ 202.184447] nixbot[2661]: {"ts": "2026-10-10T10:47:59.475Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} gitlab # [ 202.188918] nixbot[2661]: {"ts": "2026-10-10T10:47:59.479Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} gitlab # [ 202.193380] nixbot[2661]: {"ts": "2026-10-10T10:47:59.485Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} gitlab # [ 202.200340] nixbot[2661]: {"ts": "2026-10-10T10:47:59.492Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} gitlab # [ 202.210362] nixbot[2661]: {"ts": "2026-10-10T10:47:59.502Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} gitlab # [ 202.216624] nixbot[2661]: {"ts": "2026-10-10T10:47:59.508Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"} gitlab # [ 202.223645] nixbot[2661]: {"ts": "2026-10-10T10:47:59.515Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"} gitlab # [ 202.235299] nixbot[2661]: {"ts": "2026-10-10T10:47:59.527Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"} gitlab # [ 202.238810] nixbot[2661]: {"ts": "2026-10-10T10:47:59.530Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"} gitlab # [ 202.242482] nixbot[2661]: {"ts": "2026-10-10T10:47:59.534Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"} gitlab # [ 202.245760] nixbot[2661]: {"ts": "2026-10-10T10:47:59.537Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"} gitlab # [ 202.250204] nixbot[2661]: {"ts": "2026-10-10T10:47:59.542Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"} gitlab # [ 202.253348] nixbot[2661]: {"ts": "2026-10-10T10:47:59.545Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"} gitlab # [ 202.256095] nixbot[2661]: {"ts": "2026-10-10T10:47:59.548Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"} gitlab # [ 202.261226] nixbot[2661]: {"ts": "2026-10-10T10:47:59.553Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"} gitlab # [ 202.277633] nixbot[2661]: {"ts": "2026-10-10T10:47:59.569Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"} gitlab # [ 202.287307] nixbot[2661]: {"ts": "2026-10-10T10:47:59.579Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"} gitlab # [ 202.290757] nixbot[2661]: {"ts": "2026-10-10T10:47:59.582Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"} gitlab # [ 202.297862] nixbot[2661]: {"ts": "2026-10-10T10:47:59.590Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"} gitlab # [ 202.308357] nixbot[2661]: {"ts": "2026-10-10T10:47:59.600Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"} gitlab # [ 202.311223] nixbot[2661]: {"ts": "2026-10-10T10:47:59.603Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"} gitlab # [ 202.320076] nixbot[2661]: {"ts": "2026-10-10T10:47:59.612Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"} gitlab # [ 202.324451] nixbot[2661]: {"ts": "2026-10-10T10:47:59.616Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 36, "migration": "pr_approvals"} gitlab # [ 202.330126] nixbot[2661]: {"ts": "2026-10-10T10:47:59.621Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 37, "migration": "effect_when_changed"} gitlab # [ 202.657442] nixbot[2661]: {"ts": "2026-10-10T10:47:59.949Z", "level": "info", "logger": "uvicorn.error", "msg": "Started server process [2661]", "color_message": "Started server process [\u001b[36m%d\u001b[0m]"} gitlab # [ 202.659083] nixbot[2661]: {"ts": "2026-10-10T10:47:59.949Z", "level": "info", "logger": "uvicorn.error", "msg": "Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)"} gitlab # [ 202.676326] nixbot[2661]: {"ts": "2026-10-10T10:47:59.968Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:33082 - \"GET /health HTTP/1.1\" 200"} gitlab # [ 202.684668] systemd[1]: Started nixbot CI service. gitlab # [ 202.686813] systemd[1]: Reached target Multi-User System. gitlab # [ 202.688940] systemd[1]: Startup finished in 1.143s (kernel) + 3.580s (initrd) + 3min 17.964s (userspace) = 3min 22.688s. gitlab # [ 202.887268] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:00 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 210 gitlab # [ 202.901803] nixbot[2661]: {"ts": "2026-10-10T10:48:00.194Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} gitlab # [ 202.956906] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:00 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 41 gitlab # [ 202.961643] nixbot[2661]: {"ts": "2026-10-10T10:48:00.253Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} gitlab # [ 203.119789] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:00 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 917 "" "python-httpx/0.28.1" 153 gitlab # [ 203.253353] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:00 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 124 gitlab # [ 203.357816] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:00 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "python-httpx/0.28.1" 93 gitlab: (finished: waiting for unit nixbot.service, in 2.10 seconds) gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health gitlab # [ 203.398234] nixbot[2661]: {"ts": "2026-10-10T10:48:00.688Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:46250 - \"GET /health HTTP/1.1\" 200"} gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.02 seconds) subtest: project discovered and webhook registered gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 203.413282] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:00 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 917 "" "python-httpx/0.28.1" 50 gitlab # [ 203.463628] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:00 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "curl/8.21.0" 42 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.07 seconds) (finished: subtest: project discovered and webhook registered, in 0.07 seconds) subtest: push delivers the webhook gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails' gitlab # [ 203.473748] gitaly[893]: time="2026-10-10T10:48:00.765Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTHEAG52DE0677EK0SRN5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-10-10T10:48:30.446" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:00.747" grpc.time_ms=14.185 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 203.485600] gitaly[893]: time="2026-10-10T10:48:00.777Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTHEAG52DE0677EK0SRN5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:48:30.475" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:00.776" grpc.time_ms=1.057 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 203.492958] gitaly[893]: time="2026-10-10T10:48:00.785Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTHEAG52DE0677EK0SRN5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-10-10T10:48:10.049" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:00.779" grpc.time_ms=5.482 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 203.511398] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:00 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 86 gitlab # [ 203.579281] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:00 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 63 gitlab # [ 203.583188] nixbot[2661]: {"ts": "2026-10-10T10:48:00.875Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} gitlab # [ 203.779617] gitaly[893]: time="2026-10-10T10:48:01.071Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTHNNZ0FWC56ZV46GBYXM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:01.064" grpc.time_ms=6.791 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 203.781867] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:01 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 124 gitlab # [ 203.866561] gitaly[893]: time="2026-10-10T10:48:01.158Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTHSKZAFFA0B5GX25FXH7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:01.152" pid=893 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc gitlab # [ 203.867822] gitaly[893]: time="2026-10-10T10:48:01.158Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTHSKZAFFA0B5GX25FXH7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:01.152" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 203.868560] gitaly[893]: time="2026-10-10T10:48:01.159Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTHSKZAFFA0B5GX25FXH7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:01.152" grpc.time_ms=6.675 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 203.871466] gitlab-workhorse[827]: time="2026-10-10T10:48:01Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=113 gitlab # [ 203.872118] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:01 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 86 gitlab # [ 203.979621] gitaly[893]: time="2026-10-10T10:48:01.271Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTHXEVX3362ZZ5135725B grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:01.258" pid=893 remote_ip="2001:db8:1::1" request_sha=860a561774b1f737dc477bbf8e642f70ac21f724 response_bytes=223 span.kind=server system=grpc gitlab # [ 203.980107] gitaly[893]: time="2026-10-10T10:48:01.272Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1194 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTHXEVX3362ZZ5135725B grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:01.258" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 204.001184] gitaly[893]: time="2026-10-10T10:48:01.293Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1194 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTHXEVX3362ZZ5135725B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:01.258" grpc.time_ms=13.578 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 204.015110] gitlab-workhorse[827]: time="2026-10-10T10:48:01Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=223 gitlab # [ 204.015740] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:01 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 108 gitlab # [ 204.127921] gitaly[893]: time="2026-10-10T10:48:01.420Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTJ1YCSW4DN8TS81EY62K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:01.406" grpc.time_ms=13.108 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 204.155507] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:01 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 77 gitlab # [ 204.179811] gitaly[893]: time="2026-10-10T10:48:01.472Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTJ4EFNEFFCNE53GYKT9K grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:01.459" pid=893 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 204.196600] gitaly[893]: time="2026-10-10T10:48:01.472Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTJ4EFNEFFCNE53GYKT9K grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:01.459" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 204.197316] gitlab-workhorse[827]: time="2026-10-10T10:48:01Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 204.197719] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:01 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 51 gitlab # [ 204.198438] gitaly[893]: time="2026-10-10T10:48:01.472Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTJ4EFNEFFCNE53GYKT9K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:01.459" grpc.time_ms=13.218 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 204.308453] gitaly[893]: time="2026-10-10T10:48:01.600Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTJ8EX5KJ1A4ZS8D05PB5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:01.591" grpc.time_ms=8.824 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 204.336034] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:01 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 77 gitlab # [ 204.402329] gitaly[893]: time="2026-10-10T10:48:01.694Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTJB2VA1SPDQE9W7HQ8SD grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:01.676" pid=893 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc gitlab # [ 204.403960] gitaly[893]: time="2026-10-10T10:48:01.694Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1169 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTJB2VA1SPDQE9W7HQ8SD grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:01.676" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 204.446546] gitaly[893]: time="2026-10-10T10:48:01.738Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1169 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTJB2VA1SPDQE9W7HQ8SD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:01.676" grpc.time_ms=62.604 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 204.447331] gitlab-workhorse[827]: time="2026-10-10T10:48:01Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=209 gitlab # [ 204.447858] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:01 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 104 gitlab # [ 204.554962] gitaly[893]: time="2026-10-10T10:48:01.845Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:48:31.043" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:01.844" grpc.time_ms=0.6 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 204.620941] gitaly[893]: time="2026-10-10T10:48:01.913Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=648 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-10-10T10:48:31.104" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:01.904" grpc.time_ms=8.604 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 204.661418] gitaly[893]: time="2026-10-10T10:48:01.953Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:48:11.157" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:01.947" grpc.time_ms=6.127 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 204.693191] gitaly[893]: time="2026-10-10T10:48:01.985Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:48:11.189" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:01.979" grpc.time_ms=6.223 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 204.773419] gitaly[893]: time="2026-10-10T10:48:02.065Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-10T10:48:12.004" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:48:02.065" grpc.time_ms=0.382 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 204.810053] gitaly[893]: time="2026-10-10T10:48:02.102Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-10T10:48:12.029" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:02.090" grpc.time_ms=11.453 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.130070] gitaly[893]: time="2026-10-10T10:48:02.422Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:02.420" grpc.time_ms=1.506 pid=893 span.kind=server system=grpc gitlab # [ 205.191371] gitaly[893]: time="2026-10-10T10:48:02.483Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:02.480" grpc.time_ms=2.83 pid=893 span.kind=server system=grpc gitlab # [ 205.246708] gitaly[893]: time="2026-10-10T10:48:02.538Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC diskcache=7c8d7d58-013c-4ff6-8028-a79fdb62bad4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-10T10:48:12.068" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:48:02.129" pid=893 ref=refs/keep-around/bb328785a0577be29ac940af70a2aaf17ac0d917 remote_ip="2001:db8:1::1" revision=bb328785a0577be29ac940af70a2aaf17ac0d917 span.kind=server system=grpc username=root gitlab # [ 205.271600] gitaly[893]: time="2026-10-10T10:48:02.563Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=102 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=856064 command.minflt=5308 command.oublock=8 command.real_time_ms=374 command.spawn_token_fork_ms=0 command.system_time_ms=59 command.user_time_ms=43 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-10T10:48:12.068" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:48:02.129" grpc.time_ms=409.969 pid=893 ref=refs/keep-around/bb328785a0577be29ac940af70a2aaf17ac0d917 remote_ip="2001:db8:1::1" revision=bb328785a0577be29ac940af70a2aaf17ac0d917 span.kind=server system=grpc username=root gitlab # [ 205.299798] gitaly[893]: time="2026-10-10T10:48:02.592Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:48:32.090" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:02.591" grpc.time_ms=0.586 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.478270] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:02 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/bb328785a0577be29ac940af70a2aaf17ac0d917 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 974 gitlab # [ 205.487364] nixbot[2661]: {"ts": "2026-10-10T10:48:02.779Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-bb328785a057-0"} gitlab # [ 205.692049] nixbot[2661]: {"ts": "2026-10-10T10:48:02.984Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-bb328785a057-0", "argv0": "bwrap"} gitlab # [ 205.777783] systemd[1]: Started Nix Daemon. gitlab # [ 205.834829] nix-daemon[2826]: accepted connection from pid 2821, user nixbot gitlab # [ 210.517792] nix-daemon[2826]: accepted connection from pid 2869, user nixbot gitlab # [ 210.528669] nix-daemon[2826]: accepted connection from pid 2821, user nixbot gitlab # [ 215.246961] nix-daemon[2826]: accepted connection from pid 2821, user nixbot gitlab # [ 215.343661] nix-daemon[2826]: accepted connection from pid 2918, user nixbot gitlab # [ 215.395248] gitaly[893]: time="2026-10-10T10:48:12.686Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTX2G36C2VN5PKZTS2BVQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:48:42.385" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:12.685" grpc.time_ms=0.817 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 215.482619] gitaly[893]: time="2026-10-10T10:48:12.774Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTX2G36C2VN5PKZTS2BVQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:48:22.034" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:12.764" grpc.time_ms=9.914 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 215.589497] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:12 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/bb328785a0577be29ac940af70a2aaf17ac0d917 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 256 gitlab # [ 215.650500] gitaly[893]: time="2026-10-10T10:48:12.942Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTXAQG5B9VJBMH0AE3P8N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:48:42.640" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:12.941" grpc.time_ms=0.723 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 215.678419] gitaly[893]: time="2026-10-10T10:48:12.968Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTXAQG5B9VJBMH0AE3P8N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:48:22.231" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:12.962" grpc.time_ms=5.818 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 215.712083] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:13 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/bb328785a0577be29ac940af70a2aaf17ac0d917 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 116 gitlab # [ 219.346448] sidekiq-cluster[2535]: 2026-10-10T10:48:16.637Z pid=2535 tid=xu7 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2535, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>432508928} gitlab # [ 219.520816] sidekiq-cluster[2535]: W, [2026-10-10T10:48:16.812974 #2535] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 219.562438] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:16.854Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 219.911410] nix-daemon[2826]: accepted connection from pid 2987, user nixbot gitlab # [ 219.983365] gitaly[893]: time="2026-10-10T10:48:17.274Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPV1JD5FMDTG4Q5R19NBD5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:48:47.073" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:17.274" grpc.time_ms=0.729 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 220.028592] gitaly[893]: time="2026-10-10T10:48:17.320Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M4JPV1JD5FMDTG4Q5R19NBD5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:48:27.251" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:17.312" grpc.time_ms=8.491 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 220.099672] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:17 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/bb328785a0577be29ac940af70a2aaf17ac0d917 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 161 gitlab # [ 220.337077] nixbot[2661]: {"ts": "2026-10-10T10:48:17.628Z", "level": "info", "logger": "nixbot.effects_run", "msg": "effects discovered", "build_id": 1, "sha": "bb328785a0577be29ac940af70a2aaf17ac0d917", "branch": "master", "pr": null, "allowed": true, "effects_started": false, "effects": []} gitlab # [ 220.460819] gitaly[893]: time="2026-10-10T10:48:17.752Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M4JPV20KXMRQ3TX4NZTE4DBP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:17.740" grpc.time_ms=11.342 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 220.462612] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:17 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 69 gitlab # [ 220.569841] gitaly[893]: time="2026-10-10T10:48:17.862Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPV23VKCNPQ0QN6ADF1GDD grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:17.849" pid=893 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 220.571587] gitaly[893]: time="2026-10-10T10:48:17.862Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPV23VKCNPQ0QN6ADF1GDD grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:17.849" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 220.616925] gitaly[893]: time="2026-10-10T10:48:17.909Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPV23VKCNPQ0QN6ADF1GDD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:17.849" grpc.time_ms=59.423 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 220.631786] gitlab-workhorse[827]: time="2026-10-10T10:48:17Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 220.632755] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:17 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 136 gitlab # [ 220.703184] gitaly[893]: time="2026-10-10T10:48:17.995Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JPV28VCVV78BYT3SV5RG4X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:17.985" grpc.time_ms=9.495 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 220.706026] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:17 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 50 gitlab # [ 220.767417] gitaly[893]: time="2026-10-10T10:48:18.059Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPV2B1664N1V6Z18FBXX0C grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:18.052" pid=893 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc gitlab # [ 220.768950] gitaly[893]: time="2026-10-10T10:48:18.059Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPV2B1664N1V6Z18FBXX0C grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:18.052" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 220.770801] gitlab-workhorse[827]: time="2026-10-10T10:48:18Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 220.772174] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:18 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 43 gitlab # [ 220.773920] gitaly[893]: time="2026-10-10T10:48:18.060Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPV2B1664N1V6Z18FBXX0C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:18.052" grpc.time_ms=7.778 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 221.352237] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:18.643Z","message":"GitLab reliable fetch activated!"} gitlab # [ 221.354211] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:18.643Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 241.282427] postgres[3206]: [3206] ERROR: relation "table_without_models" does not exist gitlab # [ 241.284929] postgres[3206]: [3206] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 243.709271] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:41.000Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 243.711435] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:41.000Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 243.712926] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:41.001Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 243.714510] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:41.001Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 243.716057] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:41.002Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 40.52 seconds) gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master gitlab # [ 244.179060] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:41 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 128 gitlab # [ 244.271396] gitaly[893]: time="2026-10-10T10:48:41.562Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVS80JM76S42BH00M7W8Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:41.551" grpc.time_ms=11.397 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 244.272841] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:41 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.55.0" 91 gitlab # Enumerating objects: 5, done. gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. gitlab # Delta compression using up to 4 threads gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 136.00 KiB/s, done. gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) gitlab # [ 244.568663] gitaly[893]: time="2026-10-10T10:48:41.860Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=331 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVSC46XAHG01W75JXYR36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-10-10T10:49:11.652" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-10-10T10:48:41.852" grpc.time_ms=8.326 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 244.621301] gitaly[893]: time="2026-10-10T10:48:41.913Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M4JPVSC46XAHG01W75JXYR36 duration_ms=124ns method=POST pid=893 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 244.639471] gitaly[893]: time="2026-10-10T10:48:41.931Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M4JPVSC46XAHG01W75JXYR36 duration_ms=16ns method=POST pid=893 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 244.640571] gitaly[893]: time="2026-10-10T10:48:41.932Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPVSC46XAHG01W75JXYR36 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5309 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:41.788" grpc.time_ms=143.391 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 244.711308] gitaly[893]: time="2026-10-10T10:48:42.003Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPVSC46XAHG01W75JXYR36 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4779 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:42.002" grpc.time_ms=1.297 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 244.774729] gitaly[893]: time="2026-10-10T10:48:42.067Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPVSC46XAHG01W75JXYR36 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4778 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:42.065" grpc.time_ms=1.109 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 244.825654] gitaly[893]: time="2026-10-10T10:48:42.117Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPVSC46XAHG01W75JXYR36 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4780 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:42.117" grpc.time_ms=0.304 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 244.895462] gitaly[893]: time="2026-10-10T10:48:42.187Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M4JPVSC46XAHG01W75JXYR36 duration_ms=29ns method=POST pid=893 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 244.902142] gitaly[893]: time="2026-10-10T10:48:42.194Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPVSC46XAHG01W75JXYR36 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4803 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:42.157" grpc.time_ms=36.963 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 244.933114] gitaly[893]: time="2026-10-10T10:48:42.225Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M4JPVSC46XAHG01W75JXYR36 diskcache=178c70cd-24d9-434a-b3ae-ac2097c14cd7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:41.710" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 244.933895] gitaly[893]: time="2026-10-10T10:48:42.225Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=276 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=876544 command.minflt=13512 command.oublock=32 command.real_time_ms=505 command.spawn_token_fork_ms=1 command.system_time_ms=187 command.user_time_ms=89 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVSC46XAHG01W75JXYR36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:41.710" grpc.time_ms=515.349 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 244.963744] gitlab-workhorse[827]: time="2026-10-10T10:48:42Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 244.964562] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:42 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 651 gitlab # To http://gitlab/root/test-flake.git gitlab # bb32878..52b80e7 master -> master gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.98 seconds) gitlab: must succeed: git -C /tmp/test-flake rev-parse master gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=52b80e7504818909a1267a8f37adb6d0ad445c95' gitlab # [ 245.048618] nixbot[2661]: {"ts": "2026-10-10T10:48:42.340Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:42598 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=52b80e7504818909a1267a8f37adb6d0ad445c95 HTTP/1.1\" 200"} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=52b80e7504818909a1267a8f37adb6d0ad445c95', in 0.07 seconds) gitlab # [ 245.079325] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:42.370Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"192232485418fe99a9739f86","created_at":"2026-10-10T10:47:49.801Z","trace_propagation_headers":{"sentry-trace":"f1595e68c4cb401082db6ed2cc6438c6-aa3ba4924874480a","baggage":"sentry-trace_id=f1595e68c4cb401082db6ed2cc6438c6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPT5YFX9KR5AKV1T0D8K9T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:49.835Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-192232485418fe99a9739f86: start","job_status":"start","queue_duration_s":52.535677,"scheduling_latency_s":52.535756} gitlab # [ 245.276694] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:42.568Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"192232485418fe99a9739f86","created_at":"2026-10-10T10:47:49.801Z","trace_propagation_headers":{"sentry-trace":"f1595e68c4cb401082db6ed2cc6438c6-aa3ba4924874480a","baggage":"sentry-trace_id=f1595e68c4cb401082db6ed2cc6438c6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPT5YFX9KR5AKV1T0D8K9T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:49.835Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-192232485418fe99a9739f86: done: 0.198291 sec","job_status":"done","queue_duration_s":52.535677,"scheduling_latency_s":52.535756,"redis_calls":25,"redis_duration_s":0.006503,"redis_read_bytes":912,"redis_write_bytes":2805,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.003677,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001637,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001189,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":11,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.079337,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.087690475,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.198291,"completed_at":"2026-10-10T10:48:42.568Z","load_balancing_strategy":"primary","db_duration_s":0.054358,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 245.279740] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:42.570Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"e21e507145340532b3577e1a","created_at":"2026-10-10T10:47:49.844Z","trace_propagation_headers":{"sentry-trace":"f1595e68c4cb401082db6ed2cc6438c6-aa3ba4924874480a","baggage":"sentry-trace_id=f1595e68c4cb401082db6ed2cc6438c6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPT5YFX9KR5AKV1T0D8K9T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:49.848Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-e21e507145340532b3577e1a: start","job_status":"start","queue_duration_s":52.72141,"scheduling_latency_s":52.721427} gitlab # [ 245.356553] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:42.647Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"e21e507145340532b3577e1a","created_at":"2026-10-10T10:47:49.844Z","trace_propagation_headers":{"sentry-trace":"f1595e68c4cb401082db6ed2cc6438c6-aa3ba4924874480a","baggage":"sentry-trace_id=f1595e68c4cb401082db6ed2cc6438c6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPT5YFX9KR5AKV1T0D8K9T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:49.848Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-e21e507145340532b3577e1a: done: 0.077823 sec","job_status":"done","queue_duration_s":52.72141,"scheduling_latency_s":52.721427,"redis_calls":14,"redis_duration_s":0.0032300000000000002,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000711,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001445,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001074,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.048021,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.06121169,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.077823,"completed_at":"2026-10-10T10:48:42.647Z","load_balancing_strategy":"primary","db_duration_s":0.02367,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 245.369321] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:42.649Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"2018b5377e7f214d6d621398","created_at":"2026-10-10T10:47:50.395Z","trace_propagation_headers":{"sentry-trace":"f1595e68c4cb401082db6ed2cc6438c6-aa3ba4924874480a","baggage":"sentry-trace_id=f1595e68c4cb401082db6ed2cc6438c6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPT5YFX9KR5AKV1T0D8K9T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:50.400Z","job_size_bytes":5,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-2018b5377e7f214d6d621398: start","job_status":"start","queue_duration_s":52.248459,"scheduling_latency_s":52.248474} gitlab # [ 245.473560] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:42.765Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"2018b5377e7f214d6d621398","created_at":"2026-10-10T10:47:50.395Z","trace_propagation_headers":{"sentry-trace":"f1595e68c4cb401082db6ed2cc6438c6-aa3ba4924874480a","baggage":"sentry-trace_id=f1595e68c4cb401082db6ed2cc6438c6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPT5YFX9KR5AKV1T0D8K9T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:50.400Z","job_size_bytes":5,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-2018b5377e7f214d6d621398: done: 0.116073 sec","job_status":"done","queue_duration_s":52.248459,"scheduling_latency_s":52.248474,"redis_calls":12,"redis_duration_s":0.002793,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000677,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001698,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000418,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.084842,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000289423,"duration_s":0.116073,"completed_at":"2026-10-10T10:48:42.765Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.025086,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 245.476076] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:42.766Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"4a1842f2da808fa55da0f5a8","created_at":"2026-10-10T10:47:50.422Z","trace_propagation_headers":{"sentry-trace":"f1595e68c4cb401082db6ed2cc6438c6-aa3ba4924874480a","baggage":"sentry-trace_id=f1595e68c4cb401082db6ed2cc6438c6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPT5YFX9KR5AKV1T0D8K9T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:50.425Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-4a1842f2da808fa55da0f5a8: start","job_status":"start","queue_duration_s":52.341103,"scheduling_latency_s":52.341117} gitlab # [ 245.671612] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:42.963Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"4a1842f2da808fa55da0f5a8","created_at":"2026-10-10T10:47:50.422Z","trace_propagation_headers":{"sentry-trace":"f1595e68c4cb401082db6ed2cc6438c6-aa3ba4924874480a","baggage":"sentry-trace_id=f1595e68c4cb401082db6ed2cc6438c6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPT5YFX9KR5AKV1T0D8K9T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:50.425Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-4a1842f2da808fa55da0f5a8: done: 0.196772 sec","job_status":"done","queue_duration_s":52.341103,"scheduling_latency_s":52.341117,"redis_calls":11,"redis_duration_s":0.0025199999999999997,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000659,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001414,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000447,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.171,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.171,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.128039,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000292216,"duration_s":0.196772,"completed_at":"2026-10-10T10:48:42.963Z","load_balancing_strategy":"primary","db_duration_s":0.069823,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 245.673773] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:42.965Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"32ba8fc2417ba6629cb3adf3","created_at":"2026-10-10T10:47:50.429Z","trace_propagation_headers":{"sentry-trace":"f1595e68c4cb401082db6ed2cc6438c6-aa3ba4924874480a","baggage":"sentry-trace_id=f1595e68c4cb401082db6ed2cc6438c6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPT5YFX9KR5AKV1T0D8K9T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:50.434Z","job_size_bytes":89,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-32ba8fc2417ba6629cb3adf3: start","job_status":"start","queue_duration_s":52.530008,"scheduling_latency_s":52.530022} gitlab # [ 245.751154] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:42.992Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"32ba8fc2417ba6629cb3adf3","created_at":"2026-10-10T10:47:50.429Z","trace_propagation_headers":{"sentry-trace":"f1595e68c4cb401082db6ed2cc6438c6-aa3ba4924874480a","baggage":"sentry-trace_id=f1595e68c4cb401082db6ed2cc6438c6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPT5YFX9KR5AKV1T0D8K9T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-10-10T10:47:50.434Z","job_size_bytes":89,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-32ba8fc2417ba6629cb3adf3: done: 0.027114 sec","job_status":"done","queue_duration_s":52.530008,"scheduling_latency_s":52.530022,"redis_calls":11,"redis_duration_s":0.0024259999999999998,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000659,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001341,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000426,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015377,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000306464,"duration_s":0.027114,"completed_at":"2026-10-10T10:48:42.992Z","load_balancing_strategy":"primary","db_duration_s":0.005945,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 245.763748] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:42.993Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 bb328785a0577be29ac940af70a2aaf17ac0d917 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"aeb3fde567b45c8ba9623425","created_at":"2026-10-10T10:47:57.749Z","trace_propagation_headers":{"sentry-trace":"a787050b6e194fab92255b4c855536f6-25b296d0b04b4293","baggage":"sentry-trace_id=a787050b6e194fab92255b4c855536f6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPTDWVG5KNAG0F8A20MTN0","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:57.769Z","job_size_bytes":185,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-aeb3fde567b45c8ba9623425: start","job_status":"start","queue_duration_s":45.224281,"scheduling_latency_s":45.224295} gitlab # [ 245.779798] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:43.005Z","message":"changes: \"0000000000000000000000000000000000000000 bb328785a0577be29ac940af70a2aaf17ac0d917 refs/heads/master\\n\""} gitlab # [ 245.820247] gitaly[893]: time="2026-10-10T10:48:43.037Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:48:53.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:43.026" grpc.time_ms=11.748 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.838984] gitaly[893]: time="2026-10-10T10:48:43.131Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=88 command.spawn_token_fork_ms=84 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-10T10:48:53.021" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:48:43.042" grpc.time_ms=89.178 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.913654] gitaly[893]: time="2026-10-10T10:48:43.205Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-10-10T10:48:53.172" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:43.192" grpc.time_ms=12.78 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.969114] gitaly[893]: time="2026-10-10T10:48:43.261Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3931 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:43.260" grpc.time_ms=0.554 pid=893 span.kind=server system=grpc gitlab # [ 246.017408] gitaly[893]: time="2026-10-10T10:48:43.309Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3933 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:43.309" grpc.time_ms=0.541 pid=893 span.kind=server system=grpc gitlab # [ 246.044218] gitaly[893]: time="2026-10-10T10:48:43.336Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 diskcache=4f952ec6-1ec5-4040-b602-a869689a3a85 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-10T10:48:53.200" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:48:43.220" pid=893 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=52b80e7504818909a1267a8f37adb6d0ad445c95' gitlab # [ 246.045383] gitaly[893]: time="2026-10-10T10:48:43.336Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=86 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5202 command.oublock=8 command.real_time_ms=104 command.spawn_token_fork_ms=0 command.system_time_ms=51 command.user_time_ms=35 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-10T10:48:53.200" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:48:43.220" grpc.time_ms=115.756 pid=893 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 246.087303] gitaly[893]: time="2026-10-10T10:48:43.379Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T10:48:53.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:48:43.379" grpc.time_ms=0.133 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.095350] nixbot[2661]: {"ts": "2026-10-10T10:48:43.386Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:42600 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=52b80e7504818909a1267a8f37adb6d0ad445c95 HTTP/1.1\" 200"} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=52b80e7504818909a1267a8f37adb6d0ad445c95', in 0.04 seconds) gitlab # [ 246.117149] gitaly[893]: time="2026-10-10T10:48:43.409Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=936 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-10-10T10:48:53.026" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:43.396" grpc.time_ms=13.002 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.224332] gitaly[893]: time="2026-10-10T10:48:43.516Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=339 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-10-10T10:49:13.008" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:43.508" grpc.time_ms=7.559 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.245752] gitaly[893]: time="2026-10-10T10:48:43.538Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-10T10:48:53.167" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:48:43.537" grpc.time_ms=0.333 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.251227] gitaly[893]: time="2026-10-10T10:48:43.543Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=343 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-10-10T10:48:53.169" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-10-10T10:48:43.539" grpc.time_ms=3.824 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.277808] gitaly[893]: time="2026-10-10T10:48:43.570Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:48:53.191" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:43.561" grpc.time_ms=8.198 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.290144] gitaly[893]: time="2026-10-10T10:48:43.581Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:48:53.205" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:43.576" grpc.time_ms=4.976 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.314415] gitaly[893]: time="2026-10-10T10:48:43.606Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=966 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-10T10:49:13.092" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:43.592" grpc.time_ms=14.203 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.708824] gitaly[893]: time="2026-10-10T10:48:44.001Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-10-10T10:48:53.009" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:43.999" grpc.time_ms=1.425 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.772935] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:44.064Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 bb328785a0577be29ac940af70a2aaf17ac0d917 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"aeb3fde567b45c8ba9623425","created_at":"2026-10-10T10:47:57.749Z","trace_propagation_headers":{"sentry-trace":"a787050b6e194fab92255b4c855536f6-25b296d0b04b4293","baggage":"sentry-trace_id=a787050b6e194fab92255b4c855536f6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPTDWVG5KNAG0F8A20MTN0","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:47:57.769Z","job_size_bytes":185,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-aeb3fde567b45c8ba9623425: done: 1.070871 sec","job_status":"done","queue_duration_s":45.224281,"scheduling_latency_s":45.224295,"gitaly_calls":13,"gitaly_duration_s":0.343779,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.068642,"redis_read_bytes":2103,"redis_write_bytes":21750,"redis_cache_calls":15,"redis_cache_duration_s":0.003756,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.006586,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001711,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7838,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.00472,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.038896,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.012973,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.08,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.19,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.212,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.08,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.56649,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000464585,"exclusive_lock_hold_duration_s":0.026685793,"duration_s":1.070871,"completed_at":"2026-10-10T10:48:44.064Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.122937,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 246.787895] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:44.065Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"bfcb6c12e6af95159e076968","created_at":"2026-10-10T10:48:02.616Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-c822e0bedb954e0e","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:02.624Z","job_size_bytes":65,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-bfcb6c12e6af95159e076968: start","job_status":"start","queue_duration_s":41.441393,"scheduling_latency_s":41.441409} gitlab # [ 246.963336] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:44.254Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"bfcb6c12e6af95159e076968","created_at":"2026-10-10T10:48:02.616Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-c822e0bedb954e0e","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:02.624Z","job_size_bytes":65,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-bfcb6c12e6af95159e076968: done: 0.188912 sec","job_status":"done","queue_duration_s":41.441393,"scheduling_latency_s":41.441409,"redis_calls":16,"redis_duration_s":0.004438,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002492,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001447,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000499,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.104,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.097,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.06758,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000474362,"duration_s":0.188912,"completed_at":"2026-10-10T10:48:44.254Z","load_balancing_strategy":"primary","db_duration_s":0.112113,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 246.966286] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:44.257Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"6759325502536c84cf2426d3","created_at":"2026-10-10T10:48:02.625Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-c822e0bedb954e0e","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:02.631Z","job_size_bytes":65,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-6759325502536c84cf2426d3: start","job_status":"start","queue_duration_s":41.626061,"scheduling_latency_s":41.626095} gitlab # [ 247.046918] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:44.338Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"6759325502536c84cf2426d3","created_at":"2026-10-10T10:48:02.625Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-c822e0bedb954e0e","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:02.631Z","job_size_bytes":65,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-6759325502536c84cf2426d3: done: 0.081802 sec","job_status":"done","queue_duration_s":41.626061,"scheduling_latency_s":41.626095,"redis_calls":21,"redis_duration_s":0.005287,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.002448,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001492,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001347,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039936,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000302831,"duration_s":0.081802,"completed_at":"2026-10-10T10:48:44.338Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.031215,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=52b80e7504818909a1267a8f37adb6d0ad445c95' gitlab # [ 247.052054] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:44.340Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"a6c089ef2d00a21b73866d85","created_at":"2026-10-10T10:48:02.752Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-c822e0bedb954e0e","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:02.756Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a6c089ef2d00a21b73866d85: start","job_status":"start","queue_duration_s":41.58346,"scheduling_latency_s":41.583474} gitlab # [ 247.130513] nixbot[2661]: {"ts": "2026-10-10T10:48:44.422Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:42610 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=52b80e7504818909a1267a8f37adb6d0ad445c95 HTTP/1.1\" 200"} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=52b80e7504818909a1267a8f37adb6d0ad445c95', in 0.03 seconds) gitlab # [ 247.396697] gitaly[893]: time="2026-10-10T10:48:44.688Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:44.687" grpc.time_ms=0.544 pid=893 span.kind=server system=grpc gitlab # [ 247.455335] gitaly[893]: time="2026-10-10T10:48:44.747Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3890 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:44.746" grpc.time_ms=1.267 pid=893 span.kind=server system=grpc gitlab # [ 247.513876] gitaly[893]: time="2026-10-10T10:48:44.806Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:44.805" grpc.time_ms=0.579 pid=893 span.kind=server system=grpc gitlab # [ 247.544512] gitaly[893]: time="2026-10-10T10:48:44.836Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC diskcache=1bb6584b-9193-446c-95d2-43c3f3521e01 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-10T10:49:14.145" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:44.645" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.545913] gitaly[893]: time="2026-10-10T10:48:44.837Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=153 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7568 command.oublock=0 command.real_time_ms=181 command.spawn_token_fork_ms=0 command.system_time_ms=94 command.user_time_ms=59 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-10T10:49:14.145" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:44.645" grpc.time_ms=191.213 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.671569] gitaly[893]: time="2026-10-10T10:48:44.963Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-10T10:49:14.462" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:44.962" grpc.time_ms=1.042 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.711988] gitaly[893]: time="2026-10-10T10:48:45.004Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:15.003" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:45.003" grpc.time_ms=0.678 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.755136] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:45.047Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"a6c089ef2d00a21b73866d85","created_at":"2026-10-10T10:48:02.752Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-c822e0bedb954e0e","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:02.756Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2535,"job_status":"rescheduled","message":"PipelineProcessWorker JID-a6c089ef2d00a21b73866d85: rescheduled"} gitlab # [ 247.768338] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:45.060Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"a6c089ef2d00a21b73866d85","created_at":"2026-10-10T10:48:02.752Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-c822e0bedb954e0e","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:02.756Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a6c089ef2d00a21b73866d85: done: 0.719681 sec","job_status":"done","queue_duration_s":41.58346,"scheduling_latency_s":41.583474,"gitaly_calls":3,"gitaly_duration_s":0.201232,"redis_calls":120,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.058475,"redis_read_bytes":460,"redis_write_bytes":29292,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001775,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001796,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.010573,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":27,"redis_queues_duration_s":0.004536,"redis_queues_read_bytes":37,"redis_queues_write_bytes":15579,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.006739,"redis_queues_metadata_read_bytes":34,"redis_queues_metadata_write_bytes":3577,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000766,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000681,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.029834,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.067,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.191,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.195,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.04,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.283464,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.6379131979999999,"duration_s":0.719681,"completed_at":"2026-10-10T10:48:45.059Z","load_balancing_strategy":"primary","db_duration_s":0.187464,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 247.823122] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:45.062Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","bb328785a0577be29ac940af70a2aaf17ac0d917 52b80e7504818909a1267a8f37adb6d0ad445c95 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"27ca612b0a8bc87635694363","created_at":"2026-10-10T10:48:42.177Z","trace_propagation_headers":{"sentry-trace":"57ea9b650b944d58ac3842cef88d56c1-490fea6fbe754ead","baggage":"sentry-trace_id=57ea9b650b944d58ac3842cef88d56c1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPVSC46XAHG01W75JXYR36","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:42.178Z","job_size_bytes":185,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-27ca612b0a8bc87635694363: start","job_status":"start","queue_duration_s":2.883684,"scheduling_latency_s":2.883718} gitlab # [ 247.823701] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:45.076Z","message":"changes: \"bb328785a0577be29ac940af70a2aaf17ac0d917 52b80e7504818909a1267a8f37adb6d0ad445c95 refs/heads/master\\n\""} gitlab # [ 247.824225] gitaly[893]: time="2026-10-10T10:48:45.106Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVSC46XAHG01W75JXYR36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:48:55.075" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:45.096" grpc.time_ms=10.682 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.828213] gitaly[893]: time="2026-10-10T10:48:45.120Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVSC46XAHG01W75JXYR36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-10T10:48:55.089" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:48:45.110" grpc.time_ms=10.331 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.839229] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:45.126Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"512b0dc6f5d32f786cf39e8e","created_at":"2026-10-10T10:48:45.123Z","trace_propagation_headers":{"sentry-trace":"57ea9b650b944d58ac3842cef88d56c1-d658bda6ff3749ec","baggage":"sentry-trace_id=57ea9b650b944d58ac3842cef88d56c1,sentry-release=Unknown"},"correlation_id":"01M4JPVSC46XAHG01W75JXYR36","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"b1ef9403eece697b0b965132","job_size_bytes":46,"pid":2535,"job_status":"deduplicated","message":"ProjectCacheWorker JID-512b0dc6f5d32f786cf39e8e: deduplicated: until executing","deduplication.type":"until executing"} gitlab # [ 247.857739] gitaly[893]: time="2026-10-10T10:48:45.150Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVSC46XAHG01W75JXYR36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-10-10T10:48:55.120" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:45.141" grpc.time_ms=8.628 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.896777] gitaly[893]: time="2026-10-10T10:48:45.165Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=656 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVSC46XAHG01W75JXYR36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-10-10T10:49:15.157" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:45.157" grpc.time_ms=8.39 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.897716] gitaly[893]: time="2026-10-10T10:48:45.180Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=346 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVSC46XAHG01W75JXYR36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-10-10T10:48:55.156" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-10-10T10:48:45.177" grpc.time_ms=3.525 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.906196] gitaly[893]: time="2026-10-10T10:48:45.198Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVSC46XAHG01W75JXYR36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:48:55.168" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:45.188" grpc.time_ms=10.23 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.975037] gitaly[893]: time="2026-10-10T10:48:45.266Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVSC46XAHG01W75JXYR36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-10-10T10:48:55.245" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:45.265" grpc.time_ms=0.79 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.997188] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:45.289Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","bb328785a0577be29ac940af70a2aaf17ac0d917 52b80e7504818909a1267a8f37adb6d0ad445c95 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"27ca612b0a8bc87635694363","created_at":"2026-10-10T10:48:42.177Z","trace_propagation_headers":{"sentry-trace":"57ea9b650b944d58ac3842cef88d56c1-490fea6fbe754ead","baggage":"sentry-trace_id=57ea9b650b944d58ac3842cef88d56c1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPVSC46XAHG01W75JXYR36","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:42.178Z","job_size_bytes":185,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-27ca612b0a8bc87635694363: done: 0.22693 sec","job_status":"done","queue_duration_s":2.883684,"scheduling_latency_s":2.883718,"gitaly_calls":7,"gitaly_duration_s":0.066809,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.025894,"redis_read_bytes":501,"redis_write_bytes":11944,"redis_cache_calls":9,"redis_cache_duration_s":0.002826,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.001168,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5721,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.004154,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.010541,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.007205,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.016,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.016,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.121595,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000776077,"exclusive_lock_hold_duration_s":0.019582377,"duration_s":0.22693,"completed_at":"2026-10-10T10:48:45.289Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.02413,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 248.043820] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:45.335Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"b1ef9403eece697b0b965132","created_at":"2026-10-10T10:48:43.135Z","trace_propagation_headers":{"sentry-trace":"a787050b6e194fab92255b4c855536f6-7584a1a797e340b0","baggage":"sentry-trace_id=a787050b6e194fab92255b4c855536f6,sentry-release=Unknown"},"correlation_id":"01M4JPTDWVG5KNAG0F8A20MTN0","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:43.160Z","job_size_bytes":46,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-b1ef9403eece697b0b965132: start","job_status":"start","queue_duration_s":2.17463,"scheduling_latency_s":2.17468} gitlab # [ 248.104724] gitaly[893]: time="2026-10-10T10:48:45.397Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-10-10T16:48:45.394" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:48:45.394" grpc.time_ms=2.21 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=52b80e7504818909a1267a8f37adb6d0ad445c95' gitlab # [ 248.125967] gitaly[893]: time="2026-10-10T10:48:45.418Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=650 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-10-10T10:49:15.407" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:45.408" grpc.time_ms=9.949 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.165784] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:45.457Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"b1ef9403eece697b0b965132","created_at":"2026-10-10T10:48:43.135Z","trace_propagation_headers":{"sentry-trace":"a787050b6e194fab92255b4c855536f6-7584a1a797e340b0","baggage":"sentry-trace_id=a787050b6e194fab92255b4c855536f6,sentry-release=Unknown"},"correlation_id":"01M4JPTDWVG5KNAG0F8A20MTN0","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:43.160Z","job_size_bytes":46,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-b1ef9403eece697b0b965132: done: 0.122013 sec","job_status":"done","queue_duration_s":2.17463,"scheduling_latency_s":2.17468,"gitaly_calls":2,"gitaly_duration_s":0.016937,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.009477,"redis_read_bytes":69,"redis_write_bytes":4888,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.002032,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000753,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001972,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.003831,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000889,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.077071,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000436648,"duration_s":0.122013,"completed_at":"2026-10-10T10:48:45.457Z","load_balancing_strategy":"primary","db_duration_s":0.014859,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 248.187107] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:45.459Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"75957a6dc4dad7bed88e7fa3","created_at":"2026-10-10T10:48:43.180Z","trace_propagation_headers":{"sentry-trace":"a787050b6e194fab92255b4c855536f6-7584a1a797e340b0","baggage":"sentry-trace_id=a787050b6e194fab92255b4c855536f6,sentry-release=Unknown"},"correlation_id":"01M4JPTDWVG5KNAG0F8A20MTN0","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:43.186Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-75957a6dc4dad7bed88e7fa3: start","job_status":"start","queue_duration_s":2.273138,"scheduling_latency_s":2.273166} gitlab # [ 248.188067] nixbot[2661]: {"ts": "2026-10-10T10:48:45.462Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:42620 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=52b80e7504818909a1267a8f37adb6d0ad445c95 HTTP/1.1\" 200"} gitlab # [ 248.218732] gitaly[893]: time="2026-10-10T10:48:45.511Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 diskcache=bb466de8-325d-4f72-b040-7fffcc35a6d0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-10T16:48:45.482" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:45.483" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.219649] gitaly[893]: time="2026-10-10T10:48:45.511Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1284 command.oublock=0 command.real_time_ms=26 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-10T16:48:45.482" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:45.483" grpc.time_ms=28.095 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=52b80e7504818909a1267a8f37adb6d0ad445c95', in 0.12 seconds) gitlab # [ 248.261968] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:45.553Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"75957a6dc4dad7bed88e7fa3","created_at":"2026-10-10T10:48:43.180Z","trace_propagation_headers":{"sentry-trace":"a787050b6e194fab92255b4c855536f6-7584a1a797e340b0","baggage":"sentry-trace_id=a787050b6e194fab92255b4c855536f6,sentry-release=Unknown"},"correlation_id":"01M4JPTDWVG5KNAG0F8A20MTN0","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:43.186Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-75957a6dc4dad7bed88e7fa3: done: 0.093765 sec","job_status":"done","queue_duration_s":2.273138,"scheduling_latency_s":2.273166,"gitaly_calls":1,"gitaly_duration_s":0.031235,"redis_calls":15,"redis_duration_s":0.004729,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000404,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001468,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001699,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001158,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.011,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.021,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039507,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.06991978,"duration_s":0.093765,"completed_at":"2026-10-10T10:48:45.553Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015754,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.263874] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:45.555Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"2d833d2a7ab211d0d5e27fc6","created_at":"2026-10-10T10:48:43.374Z","trace_propagation_headers":{"sentry-trace":"a787050b6e194fab92255b4c855536f6-7584a1a797e340b0","baggage":"sentry-trace_id=a787050b6e194fab92255b4c855536f6,sentry-release=Unknown"},"correlation_id":"01M4JPTDWVG5KNAG0F8A20MTN0","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:43.376Z","job_size_bytes":23,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-2d833d2a7ab211d0d5e27fc6: start","job_status":"start","queue_duration_s":2.179041,"scheduling_latency_s":2.179055} gitlab # [ 248.288337] gitaly[893]: time="2026-10-10T10:48:45.580Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=647 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-10-10T10:49:15.073" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:45.574" grpc.time_ms=6.409 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.300541] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:45.592Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"2d833d2a7ab211d0d5e27fc6","created_at":"2026-10-10T10:48:43.374Z","trace_propagation_headers":{"sentry-trace":"a787050b6e194fab92255b4c855536f6-7584a1a797e340b0","baggage":"sentry-trace_id=a787050b6e194fab92255b4c855536f6,sentry-release=Unknown"},"correlation_id":"01M4JPTDWVG5KNAG0F8A20MTN0","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:43.376Z","job_size_bytes":23,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-2d833d2a7ab211d0d5e27fc6: done: 0.036887 sec","job_status":"done","queue_duration_s":2.179041,"scheduling_latency_s":2.179055,"gitaly_calls":1,"gitaly_duration_s":0.008443,"redis_calls":16,"redis_duration_s":0.003306,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000313,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000932,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001448,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000613,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021832,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036887,"completed_at":"2026-10-10T10:48:45.592Z","load_balancing_strategy":"primary","db_duration_s":0.00233,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 248.329929] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:45.594Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"24a8815bfc524ab399f8381b","created_at":"2026-10-10T10:48:43.860Z","trace_propagation_headers":{"sentry-trace":"a787050b6e194fab92255b4c855536f6-7584a1a797e340b0","baggage":"sentry-trace_id=a787050b6e194fab92255b4c855536f6,sentry-release=Unknown"},"correlation_id":"01M4JPTDWVG5KNAG0F8A20MTN0","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:31bee31b48aef260294857fa4bfbe2ced53a7790a7c977ff9010ce6607cb5593","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:43.865Z","job_size_bytes":467,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-24a8815bfc524ab399f8381b: start","job_status":"start","queue_duration_s":1.728238,"scheduling_latency_s":1.728272} gitlab # [ 248.371381] gitaly[893]: time="2026-10-10T10:48:45.616Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:15.115" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:45.616" grpc.time_ms=0.468 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.372039] gitaly[893]: time="2026-10-10T10:48:45.623Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-10T10:48:55.248" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:45.618" grpc.time_ms=4.762 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.403947] gitaly[893]: time="2026-10-10T10:48:45.695Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-10T10:49:15.194" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:45.695" grpc.time_ms=0.789 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.409368] gitaly[893]: time="2026-10-10T10:48:45.701Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=33 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-10-10T10:49:15.199" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:45.699" grpc.time_ms=1.694 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.467210] gitaly[893]: time="2026-10-10T10:48:45.759Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:48:55.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:45.753" grpc.time_ms=6.134 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.892873] gitaly[893]: time="2026-10-10T10:48:46.185Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=961 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-10T10:49:16.166" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:46.167" grpc.time_ms=17.746 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.939914] gitaly[893]: time="2026-10-10T10:48:46.232Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTDWVG5KNAG0F8A20MTN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-10-10T10:49:16.221" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:46.222" grpc.time_ms=9.735 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.998603] sidekiq-cluster[2535]: {"severity":"WARN","time":"2026-10-10T10:48:46.289Z","class":"CreatePipelineWorker","correlation_id":"01M4JPTDWVG5KNAG0F8A20MTN0","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"bb328785a0577be29ac940af70a2aaf17ac0d917","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"bb328785a0577be29ac940af70a2aaf17ac0d917","ref":"refs/heads/master"},"retry":0} gitlab # [ 249.008986] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:46.300Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"24a8815bfc524ab399f8381b","created_at":"2026-10-10T10:48:43.860Z","trace_propagation_headers":{"sentry-trace":"a787050b6e194fab92255b4c855536f6-7584a1a797e340b0","baggage":"sentry-trace_id=a787050b6e194fab92255b4c855536f6,sentry-release=Unknown"},"correlation_id":"01M4JPTDWVG5KNAG0F8A20MTN0","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:31bee31b48aef260294857fa4bfbe2ced53a7790a7c977ff9010ce6607cb5593","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:43.865Z","job_size_bytes":467,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-24a8815bfc524ab399f8381b: done: 0.706554 sec","job_status":"done","queue_duration_s":1.728238,"scheduling_latency_s":1.728272,"gitaly_calls":7,"gitaly_duration_s":0.111606,"redis_calls":51,"redis_duration_s":0.00841,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.000524,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.003448,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001623,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.001628,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001187,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.563976,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000503416,"duration_s":0.706554,"completed_at":"2026-10-10T10:48:46.300Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.03794,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 249.071501] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:46.303Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"e9ce83c0-6724-4dcb-93dc-30bb60edde86\"}"],"class":"WebHookWorker","jid":"d27b6bdc210fe174b73ee958","created_at":"2026-10-10T10:48:43.956Z","trace_propagation_headers":{"sentry-trace":"a787050b6e194fab92255b4c855536f6-7584a1a797e340b0","baggage":"sentry-trace_id=a787050b6e194fab92255b4c855536f6,sentry-release=Unknown"},"correlation_id":"01M4JPTDWVG5KNAG0F8A20MTN0","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:56c6f09202c8cc3632eb40d1f59cff242975c51800d1b08fa4e826dd21d03cf7","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:43.965Z","job_size_bytes":1751,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-d27b6bdc210fe174b73ee958: start","job_status":"start","queue_duration_s":2.338211,"scheduling_latency_s":2.338244} gitlab # [ 249.134092] nixbot[2661]: {"ts": "2026-10-10T10:48:46.426Z", "level": "info", "logger": "nixbot.webhooks", "msg": "webhook accepted", "delivery": "02816167-8d0f-4e87-8594-b5e91fc9daa2", "event": "ChangeRequest", "forge": "gitlab", "repo_id": "1", "branch": "master", "sha": "bb328785a0577be29ac940af70a2aaf17ac0d917", "pr": null} gitlab # [ 249.138863] nixbot[2661]: {"ts": "2026-10-10T10:48:46.430Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:42634 - \"POST /webhooks/gitlab HTTP/1.1\" 202"} gitlab # [ 249.164649] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:46.456Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"e9ce83c0-6724-4dcb-93dc-30bb60edde86\"}"],"class":"WebHookWorker","jid":"d27b6bdc210fe174b73ee958","created_at":"2026-10-10T10:48:43.956Z","trace_propagation_headers":{"sentry-trace":"a787050b6e194fab92255b4c855536f6-7584a1a797e340b0","baggage":"sentry-trace_id=a787050b6e194fab92255b4c855536f6,sentry-release=Unknown"},"correlation_id":"01M4JPTDWVG5KNAG0F8A20MTN0","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:56c6f09202c8cc3632eb40d1f59cff242975c51800d1b08fa4e826dd21d03cf7","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:43.965Z","job_size_bytes":1751,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-d27b6bdc210fe174b73ee958: done: 0.153036 sec","job_status":"done","queue_duration_s":2.338211,"scheduling_latency_s":2.338244,"redis_calls":19,"redis_duration_s":0.005511,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001052,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.00092,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002173,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001366,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.06208722100001296,"cpu_s":0.073426,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000471848,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.153036,"completed_at":"2026-10-10T10:48:46.456Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003261,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.215310] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:46.458Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"ad5a0b5077ea9fafe9d51d90","created_at":"2026-10-10T10:48:44.620Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-8e6a10a687c64bc3","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:44.628Z","job_size_bytes":23,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-ad5a0b5077ea9fafe9d51d90: start","job_status":"start","queue_duration_s":1.829589,"scheduling_latency_s":1.829604} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=52b80e7504818909a1267a8f37adb6d0ad445c95' gitlab # [ 249.241927] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:46.533Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"ad5a0b5077ea9fafe9d51d90","created_at":"2026-10-10T10:48:44.620Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-8e6a10a687c64bc3","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:44.628Z","job_size_bytes":23,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-ad5a0b5077ea9fafe9d51d90: done: 0.075379 sec","job_status":"done","queue_duration_s":1.829589,"scheduling_latency_s":1.829604,"redis_calls":10,"redis_duration_s":0.008513,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000464,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006977,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001072,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046215,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000957943,"duration_s":0.075379,"completed_at":"2026-10-10T10:48:46.533Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016446,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 249.262950] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:46.539Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"7bd78a5ac4b0740efe87099b","created_at":"2026-10-10T10:48:44.844Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-8e6a10a687c64bc3","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:44.851Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-7bd78a5ac4b0740efe87099b: start","job_status":"start","queue_duration_s":1.687688,"scheduling_latency_s":1.687725} gitlab # [ 249.263808] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:46 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 82 gitlab # [ 249.264461] gitaly[893]: time="2026-10-10T10:48:46.544Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVY3YJ3TF8Q17202SR1TB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:46.537" grpc.time_ms=6.731 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 249.276813] nixbot[2661]: {"ts": "2026-10-10T10:48:46.568Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:42638 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=52b80e7504818909a1267a8f37adb6d0ad445c95 HTTP/1.1\" 200"} gitlab # [ 249.282163] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:46.574Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"7bd78a5ac4b0740efe87099b","created_at":"2026-10-10T10:48:44.844Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-8e6a10a687c64bc3","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:44.851Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-7bd78a5ac4b0740efe87099b: done: 0.034948 sec","job_status":"done","queue_duration_s":1.687688,"scheduling_latency_s":1.687725,"redis_calls":10,"redis_duration_s":0.003308,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000315,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002633,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00036,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000224051,"duration_s":0.034948,"completed_at":"2026-10-10T10:48:46.574Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007821,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.342777] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:46.575Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"adf67d67b1c557ddb2231698","created_at":"2026-10-10T10:48:44.869Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-8e6a10a687c64bc3","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:44.876Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-adf67d67b1c557ddb2231698: start","job_status":"start","queue_duration_s":1.69859,"scheduling_latency_s":1.698604} gitlab # [ 249.343653] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:46.597Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"adf67d67b1c557ddb2231698","created_at":"2026-10-10T10:48:44.869Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-8e6a10a687c64bc3","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:44.876Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-adf67d67b1c557ddb2231698: done: 0.02213 sec","job_status":"done","queue_duration_s":1.69859,"scheduling_latency_s":1.698604,"redis_calls":10,"redis_duration_s":0.001313,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000255,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000809,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000249,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014427,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000232991,"duration_s":0.02213,"completed_at":"2026-10-10T10:48:46.597Z","load_balancing_strategy":"primary","db_duration_s":0.003408,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.372374] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:46.598Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"965912a9706ac6cbd97387e9","created_at":"2026-10-10T10:48:44.893Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-8e6a10a687c64bc3","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:44.897Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-965912a9706ac6cbd97387e9: start","job_status":"start","queue_duration_s":1.700348,"scheduling_latency_s":1.700361} gitlab # [ 249.378583] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:46.636Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"965912a9706ac6cbd97387e9","created_at":"2026-10-10T10:48:44.893Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-8e6a10a687c64bc3","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:44.897Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-965912a9706ac6cbd97387e9: done: 0.038509 sec","job_status":"done","queue_duration_s":1.700348,"scheduling_latency_s":1.700361,"redis_calls":13,"redis_duration_s":0.0016740000000000001,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000242,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000778,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000389,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000265,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015157,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000205054,"duration_s":0.038509,"completed_at":"2026-10-10T10:48:46.636Z","load_balancing_strategy":"primary","db_duration_s":0.003182,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.427927] gitaly[893]: time="2026-10-10T10:48:46.639Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVY6JW88F36W8GDK2Q7NW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:46.593" pid=893 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 249.428708] gitlab-workhorse[827]: time="2026-10-10T10:48:46Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 249.429565] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:46 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 94 gitlab # [ 249.430799] gitlab-workhorse[827]: time="2026-10-10T10:48:46Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=336 gitlab # [ 249.470232] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:46 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 336 "" "git/2.55.0" 67 gitlab # [ 249.470646] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:46.639Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"1e35c2bc871d9fc3f49f6cfa","created_at":"2026-10-10T10:48:44.898Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-8e6a10a687c64bc3","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:44.903Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-1e35c2bc871d9fc3f49f6cfa: start","job_status":"start","queue_duration_s":1.736676,"scheduling_latency_s":1.736691} gitlab # [ 249.471583] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:46.668Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"1e35c2bc871d9fc3f49f6cfa","created_at":"2026-10-10T10:48:44.898Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-8e6a10a687c64bc3","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:44.903Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-1e35c2bc871d9fc3f49f6cfa: done: 0.028523 sec","job_status":"done","queue_duration_s":1.736676,"scheduling_latency_s":1.736691,"redis_calls":10,"redis_duration_s":0.005347,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000588,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003509,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00125,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014326,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000205612,"duration_s":0.028523,"completed_at":"2026-10-10T10:48:46.668Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005256,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.489771] gitaly[893]: time="2026-10-10T10:48:46.639Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=43 command.spawn_token_fork_ms=38 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVY6JW88F36W8GDK2Q7NW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:46.593" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 249.501755] nixbot[2661]: {"ts": "2026-10-10T10:48:46.750Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"} gitlab # [ 249.503922] nixbot[2661]: {"ts": "2026-10-10T10:48:46.752Z", "level": "info", "logger": "nixbot.after_build", "msg": "reused build: replaying effect statuses", "build_id": 1, "sha": "bb328785a0577be29ac940af70a2aaf17ac0d917"} gitlab # [ 249.505937] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:46.670Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"4450054bec0d2a528476b115","created_at":"2026-10-10T10:48:44.903Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-8e6a10a687c64bc3","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:44.908Z","job_size_bytes":19,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-4450054bec0d2a528476b115: start","job_status":"start","queue_duration_s":1.761972,"scheduling_latency_s":1.761985} gitlab # [ 249.545674] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:46.692Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"4450054bec0d2a528476b115","created_at":"2026-10-10T10:48:44.903Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-8e6a10a687c64bc3","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:44.908Z","job_size_bytes":19,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-4450054bec0d2a528476b115: done: 0.021984 sec","job_status":"done","queue_duration_s":1.761972,"scheduling_latency_s":1.761985,"redis_calls":11,"redis_duration_s":0.004119,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000964,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002824,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000331,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011557,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000431061,"duration_s":0.021984,"completed_at":"2026-10-10T10:48:46.692Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002955,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.587211] gitaly[893]: time="2026-10-10T10:48:46.641Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=43 command.spawn_token_fork_ms=38 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVY6JW88F36W8GDK2Q7NW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:46.593" grpc.time_ms=46.216 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 249.587856] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:46.693Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"72a1bbd594aae19c122f8d5f","created_at":"2026-10-10T10:48:44.910Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-8e6a10a687c64bc3","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:44.913Z","job_size_bytes":66,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-72a1bbd594aae19c122f8d5f: start","job_status":"start","queue_duration_s":1.780092,"scheduling_latency_s":1.780105} gitlab # [ 249.588505] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:46.734Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"72a1bbd594aae19c122f8d5f","created_at":"2026-10-10T10:48:44.910Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-8e6a10a687c64bc3","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:44.913Z","job_size_bytes":66,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-72a1bbd594aae19c122f8d5f: done: 0.040747 sec","job_status":"done","queue_duration_s":1.780092,"scheduling_latency_s":1.780105,"redis_calls":10,"redis_duration_s":0.003506,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002304,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000834,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000368,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013559,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000225727,"duration_s":0.040747,"completed_at":"2026-10-10T10:48:46.733Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020594,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.644844] gitaly[893]: time="2026-10-10T10:48:46.709Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVY9KYYTKFE58WPGFSCDA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:46.697" pid=893 remote_ip="2001:db8:1::1" request_sha=10425402dc43f5ce3962c4c047d0a270e3d77174 response_bytes=336 span.kind=server system=grpc gitlab # [ 249.645617] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:46 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/bb328785a0577be29ac940af70a2aaf17ac0d917 HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 135 gitlab # [ 249.646239] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:46.735Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"93a97cdbba20f169596fd1ab","created_at":"2026-10-10T10:48:44.913Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-8e6a10a687c64bc3","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:44.917Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-93a97cdbba20f169596fd1ab: start","job_status":"start","queue_duration_s":1.818131,"scheduling_latency_s":1.818146} gitlab # [ 249.654247] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:46.767Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"93a97cdbba20f169596fd1ab","created_at":"2026-10-10T10:48:44.913Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-8e6a10a687c64bc3","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:44.917Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-93a97cdbba20f169596fd1ab: done: 0.032424 sec","job_status":"done","queue_duration_s":1.818131,"scheduling_latency_s":1.818146,"redis_calls":10,"redis_duration_s":0.003869,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000503,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002732,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000634,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019885,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000242489,"duration_s":0.032424,"completed_at":"2026-10-10T10:48:46.767Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003691,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.707620] gitaly[893]: time="2026-10-10T10:48:46.710Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1196 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVY9KYYTKFE58WPGFSCDA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:46.697" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 249.708283] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:46.770Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"343ce7675fb4cd9db67ed5c9","created_at":"2026-10-10T10:48:44.922Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-8e6a10a687c64bc3","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:44.925Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-343ce7675fb4cd9db67ed5c9: start","job_status":"start","queue_duration_s":1.845694,"scheduling_latency_s":1.845711} gitlab # [ 249.725772] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:46.806Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"343ce7675fb4cd9db67ed5c9","created_at":"2026-10-10T10:48:44.922Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-8e6a10a687c64bc3","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:44.925Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-343ce7675fb4cd9db67ed5c9: done: 0.035213 sec","job_status":"done","queue_duration_s":1.845694,"scheduling_latency_s":1.845711,"redis_calls":10,"redis_duration_s":0.004367,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000427,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002628,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001312,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014905,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000221536,"duration_s":0.035213,"completed_at":"2026-10-10T10:48:46.806Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011401,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.790690] gitaly[893]: time="2026-10-10T10:48:46.710Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1196 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVY9KYYTKFE58WPGFSCDA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:46.697" grpc.time_ms=12.72 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 249.791413] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:47 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/bb328785a0577be29ac940af70a2aaf17ac0d917 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 151 gitlab # [ 249.791856] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:46.807Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"2728e162a9257281645948cb","created_at":"2026-10-10T10:48:44.925Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-8e6a10a687c64bc3","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:44.928Z","job_size_bytes":6,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-2728e162a9257281645948cb: start","job_status":"start","queue_duration_s":1.878571,"scheduling_latency_s":1.878585} gitlab # [ 249.792502] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:46.823Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"2728e162a9257281645948cb","created_at":"2026-10-10T10:48:44.925Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-8e6a10a687c64bc3","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:44.928Z","job_size_bytes":6,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-2728e162a9257281645948cb: done: 0.015764 sec","job_status":"done","queue_duration_s":1.878571,"scheduling_latency_s":1.878585,"redis_calls":10,"redis_duration_s":0.0013340000000000001,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000283,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000795,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000256,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008993,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000202819,"duration_s":0.015764,"completed_at":"2026-10-10T10:48:46.823Z","load_balancing_strategy":"primary","db_duration_s":0.001511,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.845213] gitaly[893]: time="2026-10-10T10:48:46.811Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVYD8D3YPH2VNNASRN38K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:16.609" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:46.810" grpc.time_ms=0.469 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 249.845888] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:46.824Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"c747bf4d92f38e40161dc9ac","created_at":"2026-10-10T10:48:44.929Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-8e6a10a687c64bc3","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:44.931Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-c747bf4d92f38e40161dc9ac: start","job_status":"start","queue_duration_s":1.892704,"scheduling_latency_s":1.892718} gitlab # [ 249.856278] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:46.853Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"c747bf4d92f38e40161dc9ac","created_at":"2026-10-10T10:48:44.929Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-8e6a10a687c64bc3","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:44.931Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-c747bf4d92f38e40161dc9ac: done: 0.029293 sec","job_status":"done","queue_duration_s":1.892704,"scheduling_latency_s":1.892718,"redis_calls":10,"redis_duration_s":0.002083,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00025,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001231,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000602,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000210921,"duration_s":0.029293,"completed_at":"2026-10-10T10:48:46.853Z","load_balancing_strategy":"primary","db_duration_s":0.006402,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.900043] gitaly[893]: time="2026-10-10T10:48:46.856Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVYD8D3YPH2VNNASRN38K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:48:56.060" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:46.850" grpc.time_ms=6.055 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 249.908620] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:47 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/bb328785a0577be29ac940af70a2aaf17ac0d917 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 104 gitlab # [ 249.913135] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:46.855Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"eb1009b145c8d3af3c69a09b","created_at":"2026-10-10T10:48:44.936Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-8e6a10a687c64bc3","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:44.939Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-eb1009b145c8d3af3c69a09b: start","job_status":"start","queue_duration_s":1.915884,"scheduling_latency_s":1.915897} gitlab # [ 249.953918] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:46.902Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"eb1009b145c8d3af3c69a09b","created_at":"2026-10-10T10:48:44.936Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-8e6a10a687c64bc3","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:44.939Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-eb1009b145c8d3af3c69a09b: done: 0.04785 sec","job_status":"done","queue_duration_s":1.915884,"scheduling_latency_s":1.915897,"redis_calls":14,"redis_duration_s":0.003698,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000738,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001867,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001093,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025954,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000451174,"duration_s":0.04785,"completed_at":"2026-10-10T10:48:46.902Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008402,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.998990] gitaly[893]: time="2026-10-10T10:48:46.969Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVYHHQ0NF4NPZG6KBE22H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:16.768" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:46.969" grpc.time_ms=0.583 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 249.999774] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:47 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 47 gitlab # [ 250.000773] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:46.904Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"f360cac042204686e6274410","created_at":"2026-10-10T10:48:45.047Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-8e6a10a687c64bc3","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:45.050Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f360cac042204686e6274410: start","job_status":"start","queue_duration_s":1.853709,"scheduling_latency_s":1.853722} gitlab # [ 250.001640] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:47.168Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"c3b6d9046e00ef62b53be00c","created_at":"2026-10-10T10:48:47.166Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-6cabfc4f5f1c47af","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"f360cac042204686e6274410","job_size_bytes":3,"pid":2535,"job_status":"deduplicated","message":"PipelineProcessWorker JID-c3b6d9046e00ef62b53be00c: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60} gitlab # [ 250.003823] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:47.169Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"f360cac042204686e6274410","created_at":"2026-10-10T10:48:45.047Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-8e6a10a687c64bc3","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:45.050Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2535,"job_status":"rescheduled","message":"PipelineProcessWorker JID-f360cac042204686e6274410: rescheduled"} gitlab # [ 250.017573] gitaly[893]: time="2026-10-10T10:48:47.002Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVYHHQ0NF4NPZG6KBE22H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:48:56.206" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:46.996" grpc.time_ms=5.717 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.067088] gitaly[893]: time="2026-10-10T10:48:47.018Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:47.018" grpc.time_ms=0.305 pid=893 span.kind=server system=grpc gitlab # [ 250.067789] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:47.175Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"f360cac042204686e6274410","created_at":"2026-10-10T10:48:45.047Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-8e6a10a687c64bc3","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:45.050Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f360cac042204686e6274410: done: 0.271642 sec","job_status":"done","queue_duration_s":1.853709,"scheduling_latency_s":1.853722,"gitaly_calls":3,"gitaly_duration_s":0.101258,"redis_calls":69,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.021742,"redis_read_bytes":231,"redis_write_bytes":18471,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000628,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.00248,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.004658,"redis_queues_read_bytes":29,"redis_queues_write_bytes":12317,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.006148,"redis_queues_metadata_read_bytes":110,"redis_queues_metadata_write_bytes":3546,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000394,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00051,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.006296,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.034,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.034,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.087871,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.242092501,"duration_s":0.271642,"completed_at":"2026-10-10T10:48:47.175Z","load_balancing_strategy":"primary","db_duration_s":0.055127,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 250.117150] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:47.177Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"e60ed6564dfae83d9d137dfb","created_at":"2026-10-10T10:48:45.133Z","trace_propagation_headers":{"sentry-trace":"57ea9b650b944d58ac3842cef88d56c1-d658bda6ff3749ec","baggage":"sentry-trace_id=57ea9b650b944d58ac3842cef88d56c1,sentry-release=Unknown"},"correlation_id":"01M4JPVSC46XAHG01W75JXYR36","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"75957a6dc4dad7bed88e7fa3","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:45.136Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-e60ed6564dfae83d9d137dfb: start","job_status":"start","queue_duration_s":2.040849,"scheduling_latency_s":2.040865} gitlab # [ 250.117817] gitlab-workhorse[827]: time="2026-10-10T10:48:47Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 250.118427] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:47 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 95 gitlab # [ 250.118915] gitaly[893]: time="2026-10-10T10:48:47.044Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3890 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:47.043" grpc.time_ms=0.439 pid=893 span.kind=server system=grpc gitlab # [ 250.119383] gitaly[893]: time="2026-10-10T10:48:47.065Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:47.065" grpc.time_ms=0.281 pid=893 span.kind=server system=grpc gitlab # [ 250.119875] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:47.208Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"e60ed6564dfae83d9d137dfb","created_at":"2026-10-10T10:48:45.133Z","trace_propagation_headers":{"sentry-trace":"57ea9b650b944d58ac3842cef88d56c1-d658bda6ff3749ec","baggage":"sentry-trace_id=57ea9b650b944d58ac3842cef88d56c1,sentry-release=Unknown"},"correlation_id":"01M4JPVSC46XAHG01W75JXYR36","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"75957a6dc4dad7bed88e7fa3","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:45.136Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-e60ed6564dfae83d9d137dfb: done: 0.031542 sec","job_status":"done","queue_duration_s":2.040849,"scheduling_latency_s":2.040865,"gitaly_calls":1,"gitaly_duration_s":0.011424,"redis_calls":12,"redis_duration_s":0.001753,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00065,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000407,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000696,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014903,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.019503596,"duration_s":0.031542,"completed_at":"2026-10-10T10:48:47.208Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002597,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.192475] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:47.209Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"87077a867841cd291fd17ecc","created_at":"2026-10-10T10:48:45.235Z","trace_propagation_headers":{"sentry-trace":"57ea9b650b944d58ac3842cef88d56c1-d658bda6ff3749ec","baggage":"sentry-trace_id=57ea9b650b944d58ac3842cef88d56c1,sentry-release=Unknown"},"correlation_id":"01M4JPVSC46XAHG01W75JXYR36","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:171d828dc8ee89e3e44f6a6786dff572800fa24f3388738ad5a6968fc5564d20","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:45.238Z","job_size_bytes":467,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-87077a867841cd291fd17ecc: start","job_status":"start","queue_duration_s":1.971477,"scheduling_latency_s":1.971492} gitlab # [ 250.193233] gitaly[893]: time="2026-10-10T10:48:47.077Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC diskcache=a0cadac0-b16d-4cce-8d82-c446d5cf3b1d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-10T10:49:16.487" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:46.989" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.193934] gitaly[893]: time="2026-10-10T10:48:47.077Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=65 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7264 command.oublock=0 command.real_time_ms=79 command.spawn_token_fork_ms=0 command.system_time_ms=38 command.user_time_ms=27 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-10T10:49:16.487" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:46.989" grpc.time_ms=88.473 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.195838] gitaly[893]: time="2026-10-10T10:48:47.089Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVYPD10GVV29SD134HJVJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:17.087" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:47.088" grpc.time_ms=1.322 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.207479] gitaly[893]: time="2026-10-10T10:48:47.120Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVYPD10GVV29SD134HJVJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:48:57.055" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:47.115" grpc.time_ms=5.404 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.219545] gitaly[893]: time="2026-10-10T10:48:47.136Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-10T10:49:17.134" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:47.134" grpc.time_ms=1.804 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.230832] gitaly[893]: time="2026-10-10T10:48:47.162Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:17.161" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:47.162" grpc.time_ms=0.476 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.278253] gitaly[893]: time="2026-10-10T10:48:47.198Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVSC46XAHG01W75JXYR36 diskcache=aa6ea49d-4daf-4678-b203-baaab51b9b3f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-10T16:48:47.188" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:47.189" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.278975] gitaly[893]: time="2026-10-10T10:48:47.199Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=311 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVSC46XAHG01W75JXYR36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-10T16:48:47.188" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:47.189" grpc.time_ms=9.77 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.279591] gitaly[893]: time="2026-10-10T10:48:47.231Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVYTF9P71AYHMPV5KTPTQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:47.225" grpc.time_ms=5.444 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 250.280229] gitaly[893]: time="2026-10-10T10:48:47.289Z" level=info msg="finished unary call with code OK" catfile.duration_ms=64 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=64 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVSC46XAHG01W75JXYR36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:17.223" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:47.224" grpc.time_ms=65.029 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.284666] gitaly[893]: time="2026-10-10T10:48:47.300Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVSC46XAHG01W75JXYR36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-10T10:48:57.003" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:47.294" grpc.time_ms=6.214 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.296656] gitaly[893]: time="2026-10-10T10:48:47.327Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVSC46XAHG01W75JXYR36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-10T10:49:17.326" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:47.326" grpc.time_ms=0.476 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.307914] gitaly[893]: time="2026-10-10T10:48:47.327Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVYW1GAKSPXK6E3M3M7PF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:47.320" pid=893 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 250.350717] gitaly[893]: time="2026-10-10T10:48:47.327Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVYW1GAKSPXK6E3M3M7PF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:47.320" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 250.351598] gitaly[893]: time="2026-10-10T10:48:47.328Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVYW1GAKSPXK6E3M3M7PF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:47.320" grpc.time_ms=7.934 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 250.352229] gitaly[893]: time="2026-10-10T10:48:47.332Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=201 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVSC46XAHG01W75JXYR36 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-10-10T10:49:17.329" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:47.329" grpc.time_ms=2.374 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.391415] gitaly[893]: time="2026-10-10T10:48:47.353Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVSC46XAHG01W75JXYR36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:48:57.057" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:47.347" grpc.time_ms=5.622 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=52b80e7504818909a1267a8f37adb6d0ad445c95', in 1.18 seconds) gitlab # [ 250.430913] gitaly[893]: time="2026-10-10T10:48:47.722Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=959 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVSC46XAHG01W75JXYR36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-10T10:49:17.709" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:47.710" grpc.time_ms=12.375 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.444471] gitaly[893]: time="2026-10-10T10:48:47.736Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVSC46XAHG01W75JXYR36 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-10-10T10:49:17.724" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:47.724" grpc.time_ms=12.235 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.453699] sidekiq-cluster[2535]: {"severity":"WARN","time":"2026-10-10T10:48:47.745Z","class":"CreatePipelineWorker","correlation_id":"01M4JPVSC46XAHG01W75JXYR36","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"bb328785a0577be29ac940af70a2aaf17ac0d917","after":"52b80e7504818909a1267a8f37adb6d0ad445c95","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"52b80e7504818909a1267a8f37adb6d0ad445c95","ref":"refs/heads/master"},"retry":0} gitlab # [ 250.457661] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:47.749Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"87077a867841cd291fd17ecc","created_at":"2026-10-10T10:48:45.235Z","trace_propagation_headers":{"sentry-trace":"57ea9b650b944d58ac3842cef88d56c1-d658bda6ff3749ec","baggage":"sentry-trace_id=57ea9b650b944d58ac3842cef88d56c1,sentry-release=Unknown"},"correlation_id":"01M4JPVSC46XAHG01W75JXYR36","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:171d828dc8ee89e3e44f6a6786dff572800fa24f3388738ad5a6968fc5564d20","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:45.238Z","job_size_bytes":467,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-87077a867841cd291fd17ecc: done: 0.540087 sec","job_status":"done","queue_duration_s":1.971477,"scheduling_latency_s":1.971492,"gitaly_calls":7,"gitaly_duration_s":0.118542,"redis_calls":33,"redis_duration_s":0.003706,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000784,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000665,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.001587,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00067,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.40048,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.540087,"completed_at":"2026-10-10T10:48:47.749Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014811,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 250.466270] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:47.750Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"077a5f58-a988-415c-a8ed-cc1abd1fd4c6\"}"],"class":"WebHookWorker","jid":"cf9de4c569d8c8159875e2ed","created_at":"2026-10-10T10:48:45.253Z","trace_propagation_headers":{"sentry-trace":"57ea9b650b944d58ac3842cef88d56c1-d658bda6ff3749ec","baggage":"sentry-trace_id=57ea9b650b944d58ac3842cef88d56c1,sentry-release=Unknown"},"correlation_id":"01M4JPVSC46XAHG01W75JXYR36","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:31af98ac14d6d6ed3c7cd7887a99f2356b21008ef3719d0219a0c2dd2b196e01","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:45.255Z","job_size_bytes":1749,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-cf9de4c569d8c8159875e2ed: start","job_status":"start","queue_duration_s":2.495622,"scheduling_latency_s":2.495639} gitlab # [ 250.483135] nixbot[2661]: {"ts": "2026-10-10T10:48:47.768Z", "level": "info", "logger": "nixbot.webhooks", "msg": "webhook accepted", "delivery": "a5daa188-703a-4667-a55d-e672ee326e83", "event": "ChangeRequest", "forge": "gitlab", "repo_id": "1", "branch": "master", "sha": "52b80e7504818909a1267a8f37adb6d0ad445c95", "pr": null} gitlab # [ 250.486049] nixbot[2661]: {"ts": "2026-10-10T10:48:47.773Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:42642 - \"POST /webhooks/gitlab HTTP/1.1\" 202"} gitlab # [ 250.488200] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:47.780Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"077a5f58-a988-415c-a8ed-cc1abd1fd4c6\"}"],"class":"WebHookWorker","jid":"cf9de4c569d8c8159875e2ed","created_at":"2026-10-10T10:48:45.253Z","trace_propagation_headers":{"sentry-trace":"57ea9b650b944d58ac3842cef88d56c1-d658bda6ff3749ec","baggage":"sentry-trace_id=57ea9b650b944d58ac3842cef88d56c1,sentry-release=Unknown"},"correlation_id":"01M4JPVSC46XAHG01W75JXYR36","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:31af98ac14d6d6ed3c7cd7887a99f2356b21008ef3719d0219a0c2dd2b196e01","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:45.255Z","job_size_bytes":1749,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-cf9de4c569d8c8159875e2ed: done: 0.029323 sec","job_status":"done","queue_duration_s":2.495622,"scheduling_latency_s":2.495639,"redis_calls":14,"redis_duration_s":0.002322,"redis_read_bytes":26,"redis_write_bytes":4684,"redis_queues_calls":2,"redis_queues_duration_s":0.000449,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3728,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00138,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000493,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.008301614000004065,"cpu_s":0.01561,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.029323,"completed_at":"2026-10-10T10:48:47.780Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001202,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.543690] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:47.781Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"be7e6bc9b7e91f5aafeaeb6f","created_at":"2026-10-10T10:48:45.433Z","trace_propagation_headers":{"sentry-trace":"a787050b6e194fab92255b4c855536f6-a7abf8eb7ae84b64","baggage":"sentry-trace_id=a787050b6e194fab92255b4c855536f6,sentry-release=Unknown"},"correlation_id":"01M4JPTDWVG5KNAG0F8A20MTN0","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:45.439Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-be7e6bc9b7e91f5aafeaeb6f: start","job_status":"start","queue_duration_s":2.341487,"scheduling_latency_s":2.341503} gitlab # [ 250.549554] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:47.841Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"be7e6bc9b7e91f5aafeaeb6f","created_at":"2026-10-10T10:48:45.433Z","trace_propagation_headers":{"sentry-trace":"a787050b6e194fab92255b4c855536f6-a7abf8eb7ae84b64","baggage":"sentry-trace_id=a787050b6e194fab92255b4c855536f6,sentry-release=Unknown"},"correlation_id":"01M4JPTDWVG5KNAG0F8A20MTN0","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:45.439Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-be7e6bc9b7e91f5aafeaeb6f: done: 0.060413 sec","job_status":"done","queue_duration_s":2.341487,"scheduling_latency_s":2.341503,"redis_calls":19,"redis_duration_s":0.004184,"redis_read_bytes":22,"redis_write_bytes":4291,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000731,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.00042,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2262,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002494,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000539,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021918,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0055797600000000004,"duration_s":0.060413,"completed_at":"2026-10-10T10:48:47.841Z","load_balancing_strategy":"primary","db_duration_s":0.032139,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.609470] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:47.843Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","d27b6bdc210fe174b73ee958"],"class":"WebHooks::LogExecutionWorker","jid":"ffeb8f9590addf78bc9c47d5","created_at":"2026-10-10T10:48:46.436Z","trace_propagation_headers":{"sentry-trace":"a787050b6e194fab92255b4c855536f6-1effd532dc854d23","baggage":"sentry-trace_id=a787050b6e194fab92255b4c855536f6,sentry-release=Unknown"},"correlation_id":"01M4JPTDWVG5KNAG0F8A20MTN0","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:01883befa611d95ef89354fab5380d279b59555a1c464225d2854035d56aa201","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:46.444Z","job_size_bytes":2593,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-ffeb8f9590addf78bc9c47d5: start","job_status":"start","queue_duration_s":1.399183,"scheduling_latency_s":1.399199} gitlab # [ 250.610225] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:47.878Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","d27b6bdc210fe174b73ee958"],"class":"WebHooks::LogExecutionWorker","jid":"ffeb8f9590addf78bc9c47d5","created_at":"2026-10-10T10:48:46.436Z","trace_propagation_headers":{"sentry-trace":"a787050b6e194fab92255b4c855536f6-1effd532dc854d23","baggage":"sentry-trace_id=a787050b6e194fab92255b4c855536f6,sentry-release=Unknown"},"correlation_id":"01M4JPTDWVG5KNAG0F8A20MTN0","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:01883befa611d95ef89354fab5380d279b59555a1c464225d2854035d56aa201","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:46.444Z","job_size_bytes":2593,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-ffeb8f9590addf78bc9c47d5: done: 0.03596 sec","job_status":"done","queue_duration_s":1.399183,"scheduling_latency_s":1.399199,"redis_calls":10,"redis_duration_s":0.0026839999999999998,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000899,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001416,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000369,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020837,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000219022,"duration_s":0.03596,"completed_at":"2026-10-10T10:48:47.878Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005133,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.641137] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:47 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 76 gitlab # [ 250.642593] gitaly[893]: time="2026-10-10T10:48:47.872Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVZDKNDB9MVZHMF20JXN3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:47.865" grpc.time_ms=7.213 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 250.654640] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:47.880Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"091eddec02b7fd50cabb6114","created_at":"2026-10-10T10:48:46.981Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-6cabfc4f5f1c47af","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:46.983Z","job_size_bytes":23,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-091eddec02b7fd50cabb6114: start","job_status":"start","queue_duration_s":0.896636,"scheduling_latency_s":0.896649} gitlab # [ 250.669676] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:47.909Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"091eddec02b7fd50cabb6114","created_at":"2026-10-10T10:48:46.981Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-6cabfc4f5f1c47af","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:46.983Z","job_size_bytes":23,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-091eddec02b7fd50cabb6114: done: 0.028936 sec","job_status":"done","queue_duration_s":0.896636,"scheduling_latency_s":0.896649,"redis_calls":7,"redis_duration_s":0.0012770000000000001,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000999,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000278,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01997,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028936,"completed_at":"2026-10-10T10:48:47.908Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00507,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 250.711397] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:47.910Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"fc3ae6d5359edc678496b701","created_at":"2026-10-10T10:48:47.082Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-6cabfc4f5f1c47af","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.089Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-fc3ae6d5359edc678496b701: start","job_status":"start","queue_duration_s":0.821549,"scheduling_latency_s":0.821564} gitlab # [ 250.726959] gitlab-workhorse[827]: time="2026-10-10T10:48:47Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 250.727610] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:47 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 120 gitlab # [ 250.728128] gitaly[893]: time="2026-10-10T10:48:47.992Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVZG11JV1841BPJWE4X28 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:47.986" pid=893 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 250.736672] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.010Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"fc3ae6d5359edc678496b701","created_at":"2026-10-10T10:48:47.082Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-6cabfc4f5f1c47af","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.089Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-fc3ae6d5359edc678496b701: done: 0.099812 sec","job_status":"done","queue_duration_s":0.821549,"scheduling_latency_s":0.821564,"redis_calls":7,"redis_duration_s":0.003146,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002261,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000885,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013844,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.099812,"completed_at":"2026-10-10T10:48:48.010Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006974,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.778195] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.015Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"b3cee0b9786139471d785285","created_at":"2026-10-10T10:48:47.103Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-6cabfc4f5f1c47af","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.106Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-b3cee0b9786139471d785285: start","job_status":"start","queue_duration_s":0.909816,"scheduling_latency_s":0.909832} gitlab # [ 250.788707] gitaly[893]: time="2026-10-10T10:48:47.992Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVZG11JV1841BPJWE4X28 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:47.986" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 250.800997] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.043Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"b3cee0b9786139471d785285","created_at":"2026-10-10T10:48:47.103Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-6cabfc4f5f1c47af","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.106Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-b3cee0b9786139471d785285: done: 0.02751 sec","job_status":"done","queue_duration_s":0.909816,"scheduling_latency_s":0.909832,"redis_calls":7,"redis_duration_s":0.007037,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003358,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003679,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012274,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02751,"completed_at":"2026-10-10T10:48:48.043Z","load_balancing_strategy":"primary","db_duration_s":0.005927,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.848329] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.045Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"858e902fbf20c4bea2fb5b60","created_at":"2026-10-10T10:48:47.113Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-6cabfc4f5f1c47af","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.114Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-858e902fbf20c4bea2fb5b60: start","job_status":"start","queue_duration_s":0.930343,"scheduling_latency_s":0.930358} gitlab # [ 250.853692] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:48 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 43 gitlab # [ 250.856946] gitlab-workhorse[827]: time="2026-10-10T10:48:48Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=216 gitlab # [ 250.857378] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:48 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 48 gitlab # [ 250.896079] gitaly[893]: time="2026-10-10T10:48:47.993Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVZG11JV1841BPJWE4X28 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:47.986" grpc.time_ms=6.948 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 250.896778] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.065Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"858e902fbf20c4bea2fb5b60","created_at":"2026-10-10T10:48:47.113Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-6cabfc4f5f1c47af","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.114Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-858e902fbf20c4bea2fb5b60: done: 0.020007 sec","job_status":"done","queue_duration_s":0.930343,"scheduling_latency_s":0.930358,"redis_calls":10,"redis_duration_s":0.001878,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001207,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000406,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000265,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013102,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020007,"completed_at":"2026-10-10T10:48:48.064Z","load_balancing_strategy":"primary","db_duration_s":0.003398,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.910122] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.065Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"e6a9ea0a0530f24426389e87","created_at":"2026-10-10T10:48:47.114Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-6cabfc4f5f1c47af","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.116Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-e6a9ea0a0530f24426389e87: start","job_status":"start","queue_duration_s":0.949574,"scheduling_latency_s":0.949588} gitlab # [ 250.960974] gitaly[893]: time="2026-10-10T10:48:48.061Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVZMJRR16D5JC1BN1TR3K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:48.055" grpc.time_ms=5.491 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 250.961640] gitaly[893]: time="2026-10-10T10:48:48.110Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVZNZTFXKPKE7M4FH5K6N grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:48.098" pid=893 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc gitlab # [ 250.962473] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.090Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"e6a9ea0a0530f24426389e87","created_at":"2026-10-10T10:48:47.114Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-6cabfc4f5f1c47af","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.116Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-e6a9ea0a0530f24426389e87: done: 0.024181 sec","job_status":"done","queue_duration_s":0.949574,"scheduling_latency_s":0.949588,"redis_calls":7,"redis_duration_s":0.007106,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006391,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000715,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010966,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024181,"completed_at":"2026-10-10T10:48:48.090Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004652,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.005604] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.091Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"fdfe51211ed98b868ee8242d","created_at":"2026-10-10T10:48:47.116Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-6cabfc4f5f1c47af","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.119Z","job_size_bytes":19,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-fdfe51211ed98b868ee8242d: start","job_status":"start","queue_duration_s":0.972826,"scheduling_latency_s":0.972841} gitlab # [ 251.006511] gitaly[893]: time="2026-10-10T10:48:48.110Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1165 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVZNZTFXKPKE7M4FH5K6N grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:48.098" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 251.011966] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.108Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"fdfe51211ed98b868ee8242d","created_at":"2026-10-10T10:48:47.116Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-6cabfc4f5f1c47af","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.119Z","job_size_bytes":19,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-fdfe51211ed98b868ee8242d: done: 0.016349 sec","job_status":"done","queue_duration_s":0.972826,"scheduling_latency_s":0.972841,"redis_calls":8,"redis_duration_s":0.0017690000000000002,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001463,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000306,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010173,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016349,"completed_at":"2026-10-10T10:48:48.108Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002675,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.061651] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.109Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"6f9fa57e04c856d62920cd99","created_at":"2026-10-10T10:48:47.119Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-6cabfc4f5f1c47af","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.120Z","job_size_bytes":66,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-6f9fa57e04c856d62920cd99: start","job_status":"start","queue_duration_s":0.988395,"scheduling_latency_s":0.98841} gitlab # [ 251.067501] gitaly[893]: time="2026-10-10T10:48:48.111Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1165 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVZNZTFXKPKE7M4FH5K6N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:48.098" grpc.time_ms=12.064 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 251.115925] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.126Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"6f9fa57e04c856d62920cd99","created_at":"2026-10-10T10:48:47.119Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-6cabfc4f5f1c47af","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.120Z","job_size_bytes":66,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-6f9fa57e04c856d62920cd99: done: 0.016979 sec","job_status":"done","queue_duration_s":0.988395,"scheduling_latency_s":0.98841,"redis_calls":7,"redis_duration_s":0.002763,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000709,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002054,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009704,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016979,"completed_at":"2026-10-10T10:48:48.126Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001674,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.121602] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.127Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"d8a0ae8667e4f0b2a28ce8c0","created_at":"2026-10-10T10:48:47.121Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-6cabfc4f5f1c47af","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.122Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-d8a0ae8667e4f0b2a28ce8c0: start","job_status":"start","queue_duration_s":1.00475,"scheduling_latency_s":1.004764} gitlab # [ 251.133973] gitaly[893]: time="2026-10-10T10:48:48.181Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVZRD0EGGE2ZY9DMS83JT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:18.080" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:48.180" grpc.time_ms=0.457 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 251.146862] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.155Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"d8a0ae8667e4f0b2a28ce8c0","created_at":"2026-10-10T10:48:47.121Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-6cabfc4f5f1c47af","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.122Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-d8a0ae8667e4f0b2a28ce8c0: done: 0.028458 sec","job_status":"done","queue_duration_s":1.00475,"scheduling_latency_s":1.004764,"redis_calls":7,"redis_duration_s":0.00186,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001588,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000272,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016322,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028458,"completed_at":"2026-10-10T10:48:48.155Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006258,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.190070] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.156Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"cc1bf4527757020127d01f7d","created_at":"2026-10-10T10:48:47.122Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-6cabfc4f5f1c47af","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.123Z","job_size_bytes":6,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-cc1bf4527757020127d01f7d: start","job_status":"start","queue_duration_s":1.033199,"scheduling_latency_s":1.033217} gitlab # [ 251.230302] gitaly[893]: time="2026-10-10T10:48:48.194Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVZRD0EGGE2ZY9DMS83JT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-10-10T10:49:18.085" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:48.186" grpc.time_ms=8.035 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 251.231055] nixbot[2661]: {"ts": "2026-10-10T10:48:48.487Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-52b80e750481-5"} gitlab # [ 251.231506] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.166Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"cc1bf4527757020127d01f7d","created_at":"2026-10-10T10:48:47.122Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-6cabfc4f5f1c47af","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.123Z","job_size_bytes":6,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-cc1bf4527757020127d01f7d: done: 0.010064 sec","job_status":"done","queue_duration_s":1.033199,"scheduling_latency_s":1.033217,"redis_calls":7,"redis_duration_s":0.000994,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000731,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000263,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006387,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010064,"completed_at":"2026-10-10T10:48:48.166Z","load_balancing_strategy":"primary","db_duration_s":0.00037,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.284664] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.167Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"fb287bff124cd87237f6d051","created_at":"2026-10-10T10:48:47.124Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-6cabfc4f5f1c47af","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.125Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-fb287bff124cd87237f6d051: start","job_status":"start","queue_duration_s":1.042019,"scheduling_latency_s":1.042033} gitlab # [ 251.285481] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:48 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/52b80e7504818909a1267a8f37adb6d0ad445c95 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 339 gitlab # [ 251.285896] gitaly[893]: time="2026-10-10T10:48:48.211Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVZRD0EGGE2ZY9DMS83JT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:48:58.185" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:48.205" grpc.time_ms=5.471 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 251.286817] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.183Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"fb287bff124cd87237f6d051","created_at":"2026-10-10T10:48:47.124Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-6cabfc4f5f1c47af","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.125Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-fb287bff124cd87237f6d051: done: 0.015312 sec","job_status":"done","queue_duration_s":1.042019,"scheduling_latency_s":1.042033,"redis_calls":7,"redis_duration_s":0.001459,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001159,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0003,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010074,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015312,"completed_at":"2026-10-10T10:48:48.182Z","load_balancing_strategy":"primary","db_duration_s":0.002104,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.333802] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.183Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"04e888b34c47830fe7f57f3f","created_at":"2026-10-10T10:48:47.169Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-6cabfc4f5f1c47af","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.170Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-04e888b34c47830fe7f57f3f: start","job_status":"start","queue_duration_s":1.013269,"scheduling_latency_s":1.013283} gitlab # [ 251.334668] nixbot[2661]: {"ts": "2026-10-10T10:48:48.575Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-52b80e750481-5", "argv0": "bwrap"} gitlab # [ 251.335721] gitaly[893]: time="2026-10-10T10:48:48.240Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVZRD0EGGE2ZY9DMS83JT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-10T10:48:58.220" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:48:48.240" grpc.time_ms=0.208 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 251.339176] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.491Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"04e888b34c47830fe7f57f3f","created_at":"2026-10-10T10:48:47.169Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-6cabfc4f5f1c47af","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.170Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-04e888b34c47830fe7f57f3f: done: 0.307337 sec","job_status":"done","queue_duration_s":1.013269,"scheduling_latency_s":1.013283,"gitaly_calls":3,"gitaly_duration_s":0.162514,"redis_calls":63,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.020707,"redis_read_bytes":145,"redis_write_bytes":16719,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000356,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000598,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":20,"redis_queues_duration_s":0.002533,"redis_queues_read_bytes":25,"redis_queues_write_bytes":11225,"redis_queues_metadata_calls":18,"redis_queues_metadata_duration_s":0.011959,"redis_queues_metadata_read_bytes":28,"redis_queues_metadata_write_bytes":2886,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000164,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.002386,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.002355,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.035,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.006,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.019,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.08586,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.273767756,"duration_s":0.307337,"completed_at":"2026-10-10T10:48:48.491Z","load_balancing_strategy":"primary","db_duration_s":0.037622,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=52b80e7504818909a1267a8f37adb6d0ad445c95' gitlab # [ 251.385222] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.492Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","cf9de4c569d8c8159875e2ed"],"class":"WebHooks::LogExecutionWorker","jid":"c0ef3e21b3b587adebe674f8","created_at":"2026-10-10T10:48:47.774Z","trace_propagation_headers":{"sentry-trace":"57ea9b650b944d58ac3842cef88d56c1-b437626328894f5d","baggage":"sentry-trace_id=57ea9b650b944d58ac3842cef88d56c1,sentry-release=Unknown"},"correlation_id":"01M4JPVSC46XAHG01W75JXYR36","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8fee26ee4147e8c803c65b7b59202d8c0532a0b23ca0e799a5e969c978410b4c","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.776Z","job_size_bytes":2591,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-c0ef3e21b3b587adebe674f8: start","job_status":"start","queue_duration_s":0.716761,"scheduling_latency_s":0.716782} gitlab # [ 251.424049] gitaly[893]: time="2026-10-10T10:48:48.248Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVZRD0EGGE2ZY9DMS83JT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-10T10:48:58.224" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:48.244" grpc.time_ms=3.955 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 251.424673] gitaly[893]: time="2026-10-10T10:48:48.261Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:48.260" grpc.time_ms=0.25 pid=893 span.kind=server system=grpc gitlab # [ 251.425262] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.510Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","cf9de4c569d8c8159875e2ed"],"class":"WebHooks::LogExecutionWorker","jid":"c0ef3e21b3b587adebe674f8","created_at":"2026-10-10T10:48:47.774Z","trace_propagation_headers":{"sentry-trace":"57ea9b650b944d58ac3842cef88d56c1-b437626328894f5d","baggage":"sentry-trace_id=57ea9b650b944d58ac3842cef88d56c1,sentry-release=Unknown"},"correlation_id":"01M4JPVSC46XAHG01W75JXYR36","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8fee26ee4147e8c803c65b7b59202d8c0532a0b23ca0e799a5e969c978410b4c","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.776Z","job_size_bytes":2591,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-c0ef3e21b3b587adebe674f8: done: 0.017561 sec","job_status":"done","queue_duration_s":0.716761,"scheduling_latency_s":0.716782,"redis_calls":7,"redis_duration_s":0.000941,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00067,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000271,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011663,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017561,"completed_at":"2026-10-10T10:48:48.510Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003052,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.463926] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.511Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"4babba84da3422ecdc693f08","created_at":"2026-10-10T10:48:47.832Z","trace_propagation_headers":{"sentry-trace":"a787050b6e194fab92255b4c855536f6-bbeedfc42cea4ec9","baggage":"sentry-trace_id=a787050b6e194fab92255b4c855536f6,sentry-release=Unknown"},"correlation_id":"01M4JPTDWVG5KNAG0F8A20MTN0","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.836Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-4babba84da3422ecdc693f08: start","job_status":"start","queue_duration_s":0.675485,"scheduling_latency_s":0.6755} gitlab # [ 251.471631] gitaly[893]: time="2026-10-10T10:48:48.314Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPVZRD0EGGE2ZY9DMS83JT grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:48.276" grpc.time_ms=36.956 pid=893 span.kind=server system=grpc gitlab # [ 251.480040] gitaly[893]: time="2026-10-10T10:48:48.317Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3890 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:48.314" grpc.time_ms=2.328 pid=893 span.kind=server system=grpc gitlab # [ 251.488730] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.577Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"4babba84da3422ecdc693f08","created_at":"2026-10-10T10:48:47.832Z","trace_propagation_headers":{"sentry-trace":"a787050b6e194fab92255b4c855536f6-bbeedfc42cea4ec9","baggage":"sentry-trace_id=a787050b6e194fab92255b4c855536f6,sentry-release=Unknown"},"correlation_id":"01M4JPTDWVG5KNAG0F8A20MTN0","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:47.836Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-4babba84da3422ecdc693f08: done: 0.065389 sec","job_status":"done","queue_duration_s":0.675485,"scheduling_latency_s":0.6755,"redis_calls":11,"redis_duration_s":0.002183,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00025,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001665,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000268,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039782,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000210082,"duration_s":0.065389,"completed_at":"2026-10-10T10:48:48.576Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016711,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.537643] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.579Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"73349fc72d22f90e5a3ffa0d","created_at":"2026-10-10T10:48:48.235Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-9697952b45be48a8","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.237Z","job_size_bytes":23,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-73349fc72d22f90e5a3ffa0d: start","job_status":"start","queue_duration_s":0.342628,"scheduling_latency_s":0.342642} gitlab # [ 251.543048] gitaly[893]: time="2026-10-10T10:48:48.334Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPVZRD0EGGE2ZY9DMS83JT grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:48.334" grpc.time_ms=0.21 pid=893 span.kind=server system=grpc gitlab # [ 251.582564] gitaly[893]: time="2026-10-10T10:48:48.340Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T10:48:48.339" grpc.time_ms=0.419 pid=893 span.kind=server system=grpc gitlab # [ 251.583496] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.607Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"73349fc72d22f90e5a3ffa0d","created_at":"2026-10-10T10:48:48.235Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-9697952b45be48a8","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.237Z","job_size_bytes":23,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-73349fc72d22f90e5a3ffa0d: done: 0.027851 sec","job_status":"done","queue_duration_s":0.342628,"scheduling_latency_s":0.342642,"redis_calls":7,"redis_duration_s":0.002935,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002463,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000472,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018184,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027851,"completed_at":"2026-10-10T10:48:48.607Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004706,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 251.624596] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.608Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"df25ab0a5f3b23ab42fae9f9","created_at":"2026-10-10T10:48:48.404Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-9697952b45be48a8","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.406Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-df25ab0a5f3b23ab42fae9f9: start","job_status":"start","queue_duration_s":0.202071,"scheduling_latency_s":0.202087} gitlab # [ 251.625356] gitaly[893]: time="2026-10-10T10:48:48.343Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-10T10:49:18.240" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:48.240" operations="[repack_refs]" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 251.625987] gitaly[893]: time="2026-10-10T10:48:48.344Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-10T10:49:18.240" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:48.240" pid=893 remote_ip="2001:db8:1::1" repository_info="{false {6 702 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-10-10 10:47:49.641442815 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root gitlab # [ 251.631567] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.624Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"df25ab0a5f3b23ab42fae9f9","created_at":"2026-10-10T10:48:48.404Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-9697952b45be48a8","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.406Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-df25ab0a5f3b23ab42fae9f9: done: 0.01574 sec","job_status":"done","queue_duration_s":0.202071,"scheduling_latency_s":0.202087,"redis_calls":7,"redis_duration_s":0.000902,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000664,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000238,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010804,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01574,"completed_at":"2026-10-10T10:48:48.624Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002403,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.687081] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.625Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"b075f6b2ff216bd4d6bfae13","created_at":"2026-10-10T10:48:48.408Z","trace_propagation_headers":{"sentry-trace":"14a508b44f504785ac34b9cba92e1e2b-75e504caee494801","baggage":"sentry-trace_id=14a508b44f504785ac34b9cba92e1e2b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPVZRD0EGGE2ZY9DMS83JT","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.412Z","job_size_bytes":65,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-b075f6b2ff216bd4d6bfae13: start","job_status":"start","queue_duration_s":0.213127,"scheduling_latency_s":0.213141} gitlab # [ 251.687769] gitaly[893]: time="2026-10-10T10:48:48.345Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVZRD0EGGE2ZY9DMS83JT diskcache=b5e7499c-818f-4e70-8e4d-7e9cb53e733c grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-10T10:48:58.230" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:48:48.250" pid=893 ref=refs/keep-around/52b80e7504818909a1267a8f37adb6d0ad445c95 remote_ip="2001:db8:1::1" revision=52b80e7504818909a1267a8f37adb6d0ad445c95 span.kind=server system=grpc username=root gitlab # [ 251.693755] gitaly[893]: time="2026-10-10T10:48:48.349Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC diskcache=6273fa7a-4192-4a0c-9b8f-cad04e057638 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-10T10:49:18.240" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:48.240" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 251.732712] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.648Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"b075f6b2ff216bd4d6bfae13","created_at":"2026-10-10T10:48:48.408Z","trace_propagation_headers":{"sentry-trace":"14a508b44f504785ac34b9cba92e1e2b-75e504caee494801","baggage":"sentry-trace_id=14a508b44f504785ac34b9cba92e1e2b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPVZRD0EGGE2ZY9DMS83JT","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.412Z","job_size_bytes":65,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-b075f6b2ff216bd4d6bfae13: done: 0.023407 sec","job_status":"done","queue_duration_s":0.213127,"scheduling_latency_s":0.213141,"redis_calls":7,"redis_duration_s":0.002781,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00184,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000941,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012139,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023407,"completed_at":"2026-10-10T10:48:48.648Z","load_balancing_strategy":"primary","db_duration_s":0.005177,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 251.772949] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.649Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"f4e354de99b00b7ace575404","created_at":"2026-10-10T10:48:48.412Z","trace_propagation_headers":{"sentry-trace":"14a508b44f504785ac34b9cba92e1e2b-75e504caee494801","baggage":"sentry-trace_id=14a508b44f504785ac34b9cba92e1e2b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPVZRD0EGGE2ZY9DMS83JT","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.414Z","job_size_bytes":65,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-f4e354de99b00b7ace575404: start","job_status":"start","queue_duration_s":0.235014,"scheduling_latency_s":0.235028} gitlab # [ 251.773684] gitaly[893]: time="2026-10-10T10:48:48.396Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=43 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5191 command.oublock=8 command.real_time_ms=86 command.spawn_token_fork_ms=1 command.system_time_ms=27 command.user_time_ms=16 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVZRD0EGGE2ZY9DMS83JT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-10T10:48:58.230" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T10:48:48.250" grpc.time_ms=145.377 pid=893 ref=refs/keep-around/52b80e7504818909a1267a8f37adb6d0ad445c95 remote_ip="2001:db8:1::1" revision=52b80e7504818909a1267a8f37adb6d0ad445c95 span.kind=server system=grpc username=root gitlab # [ 251.774447] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.682Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"f4e354de99b00b7ace575404","created_at":"2026-10-10T10:48:48.412Z","trace_propagation_headers":{"sentry-trace":"14a508b44f504785ac34b9cba92e1e2b-75e504caee494801","baggage":"sentry-trace_id=14a508b44f504785ac34b9cba92e1e2b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPVZRD0EGGE2ZY9DMS83JT","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.414Z","job_size_bytes":65,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-f4e354de99b00b7ace575404: done: 0.032401 sec","job_status":"done","queue_duration_s":0.235014,"scheduling_latency_s":0.235028,"redis_calls":10,"redis_duration_s":0.0014320000000000001,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000797,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000635,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019967,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032401,"completed_at":"2026-10-10T10:48:48.682Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009524,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.823026] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.683Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"ded423ff5bcc5552079bb5ff","created_at":"2026-10-10T10:48:48.416Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-9697952b45be48a8","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.420Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-ded423ff5bcc5552079bb5ff: start","job_status":"start","queue_duration_s":0.262809,"scheduling_latency_s":0.262824} gitlab # [ 251.823643] gitaly[893]: time="2026-10-10T10:48:48.396Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=64 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7320 command.oublock=0 command.real_time_ms=102 command.spawn_token_fork_ms=0 command.system_time_ms=36 command.user_time_ms=28 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-10T10:49:18.240" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:48.240" grpc.time_ms=155.463 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 251.863800] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.699Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"ded423ff5bcc5552079bb5ff","created_at":"2026-10-10T10:48:48.416Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-9697952b45be48a8","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.420Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-ded423ff5bcc5552079bb5ff: done: 0.016696 sec","job_status":"done","queue_duration_s":0.262809,"scheduling_latency_s":0.262824,"redis_calls":7,"redis_duration_s":0.0009400000000000001,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000702,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000238,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011447,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016696,"completed_at":"2026-10-10T10:48:48.699Z","load_balancing_strategy":"primary","db_duration_s":0.00284,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.868424] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.700Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"d587e5143ba6c84f13be44b4","created_at":"2026-10-10T10:48:48.428Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-9697952b45be48a8","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.433Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-d587e5143ba6c84f13be44b4: start","job_status":"start","queue_duration_s":0.267358,"scheduling_latency_s":0.267373} gitlab # [ 251.914777] gitaly[893]: time="2026-10-10T10:48:48.404Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVZRD0EGGE2ZY9DMS83JT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:18.102" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:48.402" grpc.time_ms=1.932 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 251.915536] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.718Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"d587e5143ba6c84f13be44b4","created_at":"2026-10-10T10:48:48.428Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-9697952b45be48a8","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.433Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-d587e5143ba6c84f13be44b4: done: 0.017866 sec","job_status":"done","queue_duration_s":0.267358,"scheduling_latency_s":0.267373,"redis_calls":10,"redis_duration_s":0.001319,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000715,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000364,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00024,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012505,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017866,"completed_at":"2026-10-10T10:48:48.718Z","load_balancing_strategy":"primary","db_duration_s":0.002888,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.930789] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.719Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"463f24ee934c57048746c98a","created_at":"2026-10-10T10:48:48.433Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-9697952b45be48a8","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.434Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-463f24ee934c57048746c98a: start","job_status":"start","queue_duration_s":0.28437,"scheduling_latency_s":0.284384} gitlab # [ 251.968515] gitaly[893]: time="2026-10-10T10:48:48.411Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-10T10:49:18.240" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:48.240" pid=893 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root gitlab # [ 251.969227] gitaly[893]: time="2026-10-10T10:48:48.412Z" level=info msg="optimized repository" clean_stale_data_time=15.605056ms clean_work_trees_time="75.15µs" commit_graph_time="3.911µs" correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="4.191µs" pid=893 prune_time="3.631µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="6.146µs" storage_name=default system=housekeeping total_time=15.796979ms gitlab # [ 251.969669] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.734Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"463f24ee934c57048746c98a","created_at":"2026-10-10T10:48:48.433Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-9697952b45be48a8","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.434Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-463f24ee934c57048746c98a: done: 0.015135 sec","job_status":"done","queue_duration_s":0.28437,"scheduling_latency_s":0.284384,"redis_calls":7,"redis_duration_s":0.000941,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000689,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000252,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010236,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015135,"completed_at":"2026-10-10T10:48:48.734Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002325,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.023041] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.735Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"34b8d10ac7d597bfdf98f08f","created_at":"2026-10-10T10:48:48.435Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-9697952b45be48a8","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.438Z","job_size_bytes":19,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-34b8d10ac7d597bfdf98f08f: start","job_status":"start","queue_duration_s":0.297378,"scheduling_latency_s":0.297391} gitlab # [ 252.023699] gitaly[893]: time="2026-10-10T10:48:48.412Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-10T10:49:18.240" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:48.240" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.024203] gitaly[893]: time="2026-10-10T10:48:48.456Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-10T10:49:18.455" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:48.455" grpc.time_ms=0.589 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.067109] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.749Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"34b8d10ac7d597bfdf98f08f","created_at":"2026-10-10T10:48:48.435Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-9697952b45be48a8","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.438Z","job_size_bytes":19,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-34b8d10ac7d597bfdf98f08f: done: 0.014511 sec","job_status":"done","queue_duration_s":0.297378,"scheduling_latency_s":0.297391,"redis_calls":8,"redis_duration_s":0.001088,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000841,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000247,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009663,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014511,"completed_at":"2026-10-10T10:48:48.749Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002245,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.106670] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.750Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"e0d06977bfd261926c700234","created_at":"2026-10-10T10:48:48.438Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-9697952b45be48a8","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.441Z","job_size_bytes":66,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-e0d06977bfd261926c700234: start","job_status":"start","queue_duration_s":0.309439,"scheduling_latency_s":0.309453} gitlab # [ 252.107576] gitaly[893]: time="2026-10-10T10:48:48.479Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPTJG366Z8DNA7SE3MDNTC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:18.478" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:48.479" grpc.time_ms=0.465 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.108281] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.770Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"e0d06977bfd261926c700234","created_at":"2026-10-10T10:48:48.438Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-9697952b45be48a8","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.441Z","job_size_bytes":66,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-e0d06977bfd261926c700234: done: 0.019196 sec","job_status":"done","queue_duration_s":0.309439,"scheduling_latency_s":0.309453,"redis_calls":7,"redis_duration_s":0.006307,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005582,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000725,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009647,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019196,"completed_at":"2026-10-10T10:48:48.770Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001782,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.154772] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.772Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"16f49d2c58d9b38f393624d4","created_at":"2026-10-10T10:48:48.442Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-9697952b45be48a8","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.444Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-16f49d2c58d9b38f393624d4: start","job_status":"start","queue_duration_s":0.328119,"scheduling_latency_s":0.328134} gitlab # [ 252.155810] gitaly[893]: time="2026-10-10T10:48:48.863Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPVZRD0EGGE2ZY9DMS83JT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-10T10:49:18.862" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:48.862" grpc.time_ms=0.68 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.162870] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.792Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"16f49d2c58d9b38f393624d4","created_at":"2026-10-10T10:48:48.442Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-9697952b45be48a8","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.444Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-16f49d2c58d9b38f393624d4: done: 0.019777 sec","job_status":"done","queue_duration_s":0.328119,"scheduling_latency_s":0.328134,"redis_calls":7,"redis_duration_s":0.003044,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002075,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000969,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010515,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019777,"completed_at":"2026-10-10T10:48:48.792Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004155,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.207762] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.793Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"02aa5610b6cb3b58ff2afb8e","created_at":"2026-10-10T10:48:48.445Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-9697952b45be48a8","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.446Z","job_size_bytes":6,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-02aa5610b6cb3b58ff2afb8e: start","job_status":"start","queue_duration_s":0.347258,"scheduling_latency_s":0.347293} gitlab # [ 252.215106] gitaly[893]: time="2026-10-10T10:48:48.885Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPVZRD0EGGE2ZY9DMS83JT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:18.884" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:48.884" grpc.time_ms=0.536 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.261849] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.802Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"02aa5610b6cb3b58ff2afb8e","created_at":"2026-10-10T10:48:48.445Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-9697952b45be48a8","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.446Z","job_size_bytes":6,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-02aa5610b6cb3b58ff2afb8e: done: 0.008666 sec","job_status":"done","queue_duration_s":0.347258,"scheduling_latency_s":0.347293,"redis_calls":7,"redis_duration_s":0.000879,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000632,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000247,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005521,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008666,"completed_at":"2026-10-10T10:48:48.802Z","load_balancing_strategy":"primary","db_duration_s":0.000236,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.263376] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.803Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"93bbc6834db4c7a93b60d76b","created_at":"2026-10-10T10:48:48.446Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-9697952b45be48a8","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.447Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-93bbc6834db4c7a93b60d76b: start","job_status":"start","queue_duration_s":0.355531,"scheduling_latency_s":0.355545} gitlab # [ 252.277895] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.816Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"93bbc6834db4c7a93b60d76b","created_at":"2026-10-10T10:48:48.446Z","trace_propagation_headers":{"sentry-trace":"4d623347097f4af2b9a1c49fe79b7b8f-9697952b45be48a8","baggage":"sentry-trace_id=4d623347097f4af2b9a1c49fe79b7b8f,sentry-release=Unknown"},"correlation_id":"01M4JPTJG366Z8DNA7SE3MDNTC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.447Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-93bbc6834db4c7a93b60d76b: done: 0.013726 sec","job_status":"done","queue_duration_s":0.355531,"scheduling_latency_s":0.355545,"redis_calls":7,"redis_duration_s":0.00102,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000744,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000276,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009225,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013726,"completed_at":"2026-10-10T10:48:48.816Z","load_balancing_strategy":"primary","db_duration_s":0.001946,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.318826] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.817Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"a36eb972aeb68dcce1968b05","created_at":"2026-10-10T10:48:48.466Z","trace_propagation_headers":{"sentry-trace":"14a508b44f504785ac34b9cba92e1e2b-75e504caee494801","baggage":"sentry-trace_id=14a508b44f504785ac34b9cba92e1e2b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPVZRD0EGGE2ZY9DMS83JT","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.470Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a36eb972aeb68dcce1968b05: start","job_status":"start","queue_duration_s":0.347819,"scheduling_latency_s":0.347831} gitlab # [ 252.329464] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.896Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"a36eb972aeb68dcce1968b05","created_at":"2026-10-10T10:48:48.466Z","trace_propagation_headers":{"sentry-trace":"14a508b44f504785ac34b9cba92e1e2b-75e504caee494801","baggage":"sentry-trace_id=14a508b44f504785ac34b9cba92e1e2b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JPVZRD0EGGE2ZY9DMS83JT","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.470Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a36eb972aeb68dcce1968b05: done: 0.078227 sec","job_status":"done","queue_duration_s":0.347819,"scheduling_latency_s":0.347831,"gitaly_calls":2,"gitaly_duration_s":0.004447,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.004234,"redis_read_bytes":79,"redis_write_bytes":5122,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000583,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000765,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000189,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000953,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000144,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000419,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000598,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055966,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.058043131,"duration_s":0.078227,"completed_at":"2026-10-10T10:48:48.896Z","load_balancing_strategy":"primary","db_duration_s":0.017494,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 252.377069] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.897Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"31a4a218264ddfec7712a6db","created_at":"2026-10-10T10:48:48.852Z","trace_propagation_headers":{"sentry-trace":"14a508b44f504785ac34b9cba92e1e2b-e4b0c7268b0e41c5","baggage":"sentry-trace_id=14a508b44f504785ac34b9cba92e1e2b,sentry-release=Unknown"},"correlation_id":"01M4JPVZRD0EGGE2ZY9DMS83JT","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.853Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-31a4a218264ddfec7712a6db: start","job_status":"start","queue_duration_s":0.043325,"scheduling_latency_s":0.04334} gitlab # [ 252.419061] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:48.920Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"31a4a218264ddfec7712a6db","created_at":"2026-10-10T10:48:48.852Z","trace_propagation_headers":{"sentry-trace":"14a508b44f504785ac34b9cba92e1e2b-e4b0c7268b0e41c5","baggage":"sentry-trace_id=14a508b44f504785ac34b9cba92e1e2b,sentry-release=Unknown"},"correlation_id":"01M4JPVZRD0EGGE2ZY9DMS83JT","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:48.853Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-31a4a218264ddfec7712a6db: done: 0.023788 sec","job_status":"done","queue_duration_s":0.043325,"scheduling_latency_s":0.04334,"redis_calls":7,"redis_duration_s":0.001043,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000779,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000264,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01726,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023788,"completed_at":"2026-10-10T10:48:48.920Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003366,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.616435] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 2182486086 wd_nsec: 2182485997 gitlab # [ 252.491112] nixbot[2661]: {"ts": "2026-10-10T10:48:49.782Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:42646 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=52b80e7504818909a1267a8f37adb6d0ad445c95 HTTP/1.1\" 200"} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=52b80e7504818909a1267a8f37adb6d0ad445c95', in 1.08 seconds) (finished: subtest: push delivers the webhook, in 49.05 seconds) subtest: eval and build post commit statuses gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/52b80e7504818909a1267a8f37adb6d0ad445c95/statuses gitlab # [ 252.529492] nix-daemon[2826]: accepted connection from pid 3657, user nixbot gitlab # [ 252.609099] gitaly[893]: time="2026-10-10T10:48:49.900Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW1DJQPD09ACD5ETSZQF9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:19.599" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:49.900" grpc.time_ms=0.628 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.627646] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/52b80e7504818909a1267a8f37adb6d0ad445c95/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 76 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/52b80e7504818909a1267a8f37adb6d0ad445c95/statuses, in 0.13 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/52b80e7504818909a1267a8f37adb6d0ad445c95/statuses gitlab # [ 253.739210] gitaly[893]: time="2026-10-10T10:48:51.030Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW2H0NSK3NRH80F3G9RV5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:21.029" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:51.029" grpc.time_ms=0.82 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 253.804264] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:51 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/52b80e7504818909a1267a8f37adb6d0ad445c95/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 118 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/52b80e7504818909a1267a8f37adb6d0ad445c95/statuses, in 0.16 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/52b80e7504818909a1267a8f37adb6d0ad445c95/statuses gitlab # [ 254.903562] gitaly[893]: time="2026-10-10T10:48:52.195Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW3N9K0GTE5002255MVDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:22.193" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:52.194" grpc.time_ms=0.84 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 254.929145] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:52 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/52b80e7504818909a1267a8f37adb6d0ad445c95/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 83 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/52b80e7504818909a1267a8f37adb6d0ad445c95/statuses, in 0.12 seconds) gitlab # [ 255.269543] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:52.560Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"7c5c801b23bf287527ca90e2","created_at":"2026-10-10T10:48:52.544Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c1af5979ae1b36a1cb31b7604c6ea8d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.558Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-7c5c801b23bf287527ca90e2: start","job_status":"start","queue_duration_s":0.002606,"scheduling_latency_s":0.002644} gitlab # [ 255.336934] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:52.628Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"7c5c801b23bf287527ca90e2","created_at":"2026-10-10T10:48:52.544Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c1af5979ae1b36a1cb31b7604c6ea8d0","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.558Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-7c5c801b23bf287527ca90e2: done: 0.067936 sec","job_status":"done","queue_duration_s":0.002606,"scheduling_latency_s":0.002644,"redis_calls":12,"redis_duration_s":0.016839,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004888,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005276,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.006675,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018306,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000903188,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.067936,"completed_at":"2026-10-10T10:48:52.628Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000686,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.384542] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:52.634Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"906f08df86b7b1ae1cdba105","created_at":"2026-10-10T10:48:52.575Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"964f37a31f85f53ed6111dae4bd3885a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.590Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-906f08df86b7b1ae1cdba105: start","job_status":"start","queue_duration_s":0.044572,"scheduling_latency_s":0.04461} gitlab # [ 255.386002] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:52.673Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 255.434433] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:52.726Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"906f08df86b7b1ae1cdba105","created_at":"2026-10-10T10:48:52.575Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"964f37a31f85f53ed6111dae4bd3885a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.590Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-906f08df86b7b1ae1cdba105: done: 0.091383 sec","job_status":"done","queue_duration_s":0.044572,"scheduling_latency_s":0.04461,"redis_calls":9,"redis_duration_s":0.016911,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001881,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.014046,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000984,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018427,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.091383,"completed_at":"2026-10-10T10:48:52.726Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00253,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.477460] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:52.769Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"c481a8d2fe01134fbe75e53f","created_at":"2026-10-10T10:48:52.605Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"6aab61db3e5e8073d650691991554186","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.616Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-c481a8d2fe01134fbe75e53f: start","job_status":"start","queue_duration_s":0.152616,"scheduling_latency_s":0.152658} gitlab # [ 255.532858] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:52.824Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"c481a8d2fe01134fbe75e53f","created_at":"2026-10-10T10:48:52.605Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"6aab61db3e5e8073d650691991554186","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.616Z","job_size_bytes":3,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-c481a8d2fe01134fbe75e53f: done: 0.055136 sec","job_status":"done","queue_duration_s":0.152616,"scheduling_latency_s":0.152658,"redis_calls":10,"redis_duration_s":0.014365,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003405,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006183,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004777,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015049,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000767136,"duration_s":0.055136,"completed_at":"2026-10-10T10:48:52.824Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001642,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.572911] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:52.864Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"dabe74104a5463907a1ca165","created_at":"2026-10-10T10:48:52.629Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.639Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-dabe74104a5463907a1ca165: start","job_status":"start","queue_duration_s":0.225147,"scheduling_latency_s":0.225187} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/52b80e7504818909a1267a8f37adb6d0ad445c95/statuses gitlab # [ 255.910271] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.201Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"dabe74104a5463907a1ca165","created_at":"2026-10-10T10:48:52.629Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.639Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-dabe74104a5463907a1ca165: done: 0.33702 sec","job_status":"done","queue_duration_s":0.225147,"scheduling_latency_s":0.225187,"redis_calls":65,"redis_duration_s":0.16321,"redis_read_bytes":114,"redis_write_bytes":41332,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.003643,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":656,"redis_queues_calls":2,"redis_queues_duration_s":0.001543,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37035,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.117629,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3374,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.040395,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":267,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.126399,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000928889,"duration_s":0.33702,"completed_at":"2026-10-10T10:48:53.201Z","load_balancing_strategy":"primary","db_duration_s":0.001999,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.003149] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.294Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"e03796d311748b3de145ff10","created_at":"2026-10-10T10:48:52.660Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ec99f11602732e40b7be9e383e9574a0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-10T10:48:52.767Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-e03796d311748b3de145ff10: start","job_status":"start","queue_duration_s":0.527621,"scheduling_latency_s":0.527644} gitlab # [ 256.044758] gitaly[893]: time="2026-10-10T10:48:53.336Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW4S2BR2YDHW5FC8R1921 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:23.335" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:53.335" grpc.time_ms=0.529 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 256.066428] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:53 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/52b80e7504818909a1267a8f37adb6d0ad445c95/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 75 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/52b80e7504818909a1267a8f37adb6d0ad445c95/statuses, in 0.13 seconds) gitlab # [ 256.074133] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.365Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"e03796d311748b3de145ff10","created_at":"2026-10-10T10:48:52.660Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ec99f11602732e40b7be9e383e9574a0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-10T10:48:52.767Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-e03796d311748b3de145ff10: done: 0.070754 sec","job_status":"done","queue_duration_s":0.527621,"scheduling_latency_s":0.527644,"redis_calls":9,"redis_duration_s":0.015775,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001905,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.01207,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0018,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020738,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.070754,"completed_at":"2026-10-10T10:48:53.365Z","load_balancing_strategy":"primary","db_duration_s":0.019856,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.119351] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.411Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"f17ba50a534a4636d2f65944","created_at":"2026-10-10T10:48:52.792Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"de69f851b1f44ab581cb9894241750dc","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.805Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-f17ba50a534a4636d2f65944: start","job_status":"start","queue_duration_s":0.605559,"scheduling_latency_s":0.605602} gitlab # [ 256.146766] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.438Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"f17ba50a534a4636d2f65944","created_at":"2026-10-10T10:48:52.792Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"de69f851b1f44ab581cb9894241750dc","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.805Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-f17ba50a534a4636d2f65944: done: 0.027929 sec","job_status":"done","queue_duration_s":0.605559,"scheduling_latency_s":0.605602,"redis_calls":12,"redis_duration_s":0.006887,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000497,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000617,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005272,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000501,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009158,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027741,"duration_s":0.027929,"completed_at":"2026-10-10T10:48:53.438Z","load_balancing_strategy":"primary","db_duration_s":0.000629,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.148508] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.440Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"176a5296f3903b0c3e3a62ee","created_at":"2026-10-10T10:48:52.818Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8f1fe23444e6bc1cc8909c17e3f2da0d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.826Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-176a5296f3903b0c3e3a62ee: start","job_status":"start","queue_duration_s":0.613704,"scheduling_latency_s":0.613718} gitlab # [ 256.188058] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.448Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 256.188753] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.453Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"176a5296f3903b0c3e3a62ee","created_at":"2026-10-10T10:48:52.818Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8f1fe23444e6bc1cc8909c17e3f2da0d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.826Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-176a5296f3903b0c3e3a62ee: done: 0.013168 sec","job_status":"done","queue_duration_s":0.613704,"scheduling_latency_s":0.613718,"redis_calls":9,"redis_duration_s":0.001911,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000434,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001062,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000415,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007075,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013168,"completed_at":"2026-10-10T10:48:53.453Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000401,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.230857] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.454Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"fc42cc252ebafecc479e2794","created_at":"2026-10-10T10:48:52.921Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ec3cc75adb628597b465dc8231bdd4d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-10T10:48:52.937Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-fc42cc252ebafecc479e2794: start","job_status":"start","queue_duration_s":0.516908,"scheduling_latency_s":0.516923} gitlab # [ 256.231716] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.462Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 256.232637] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.466Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"fc42cc252ebafecc479e2794","created_at":"2026-10-10T10:48:52.921Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ec3cc75adb628597b465dc8231bdd4d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-10T10:48:52.937Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-fc42cc252ebafecc479e2794: done: 0.012478 sec","job_status":"done","queue_duration_s":0.516908,"scheduling_latency_s":0.516923,"redis_calls":9,"redis_duration_s":0.0019,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000479,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001008,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000413,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006711,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012478,"completed_at":"2026-10-10T10:48:53.466Z","load_balancing_strategy":"primary","db_duration_s":0.000461,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.252376] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.468Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"51083fcb1b23e507e34612c1","created_at":"2026-10-10T10:48:52.957Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"12e1c653b41920e6ed7db5dc7329d0b9","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.976Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-51083fcb1b23e507e34612c1: start","job_status":"start","queue_duration_s":0.492581,"scheduling_latency_s":0.492596} gitlab # [ 256.266575] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.490Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"51083fcb1b23e507e34612c1","created_at":"2026-10-10T10:48:52.957Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"12e1c653b41920e6ed7db5dc7329d0b9","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:52.976Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-51083fcb1b23e507e34612c1: done: 0.021408 sec","job_status":"done","queue_duration_s":0.492581,"scheduling_latency_s":0.492596,"redis_calls":12,"redis_duration_s":0.002735,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000451,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001295,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000989,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012087,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.006892775,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.021408,"completed_at":"2026-10-10T10:48:53.490Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003504,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 256.319429] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.491Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"77066b8f08e102f85948e09a","created_at":"2026-10-10T10:48:53.001Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f8a31826137b01b1df387f98d6859fb4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.015Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-77066b8f08e102f85948e09a: start","job_status":"start","queue_duration_s":0.47622,"scheduling_latency_s":0.476233} gitlab # [ 256.320679] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.505Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"77066b8f08e102f85948e09a","created_at":"2026-10-10T10:48:53.001Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f8a31826137b01b1df387f98d6859fb4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.015Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-77066b8f08e102f85948e09a: done: 0.014511 sec","job_status":"done","queue_duration_s":0.47622,"scheduling_latency_s":0.476233,"redis_calls":11,"redis_duration_s":0.002332,"redis_read_bytes":9,"redis_write_bytes":1386,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000868,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001021,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000443,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008208,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014511,"completed_at":"2026-10-10T10:48:53.505Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000874,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.368095] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.507Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"2ce5a4b24fd0ab9c82a5cbdf","created_at":"2026-10-10T10:48:53.025Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3282491e5311b3d124f9d38ddb7504ee","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.038Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-2ce5a4b24fd0ab9c82a5cbdf: start","job_status":"start","queue_duration_s":0.468736,"scheduling_latency_s":0.468749} gitlab # [ 256.407950] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.554Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"2ce5a4b24fd0ab9c82a5cbdf","created_at":"2026-10-10T10:48:53.025Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3282491e5311b3d124f9d38ddb7504ee","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.038Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-2ce5a4b24fd0ab9c82a5cbdf: done: 0.047074 sec","job_status":"done","queue_duration_s":0.468736,"scheduling_latency_s":0.468749,"redis_calls":11,"redis_duration_s":0.002651,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000439,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001113,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001099,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021174,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000516546,"exclusive_lock_hold_duration_s":0.022248917,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.047074,"completed_at":"2026-10-10T10:48:53.554Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019872,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.412578] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.555Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"829fc137343d6de4170c6e4d","created_at":"2026-10-10T10:48:53.049Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"950994d6b63e75cde0a67966cfada337","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.061Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-829fc137343d6de4170c6e4d: start","job_status":"start","queue_duration_s":0.49423,"scheduling_latency_s":0.494245} gitlab # [ 256.521905] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.813Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"829fc137343d6de4170c6e4d","created_at":"2026-10-10T10:48:53.049Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"950994d6b63e75cde0a67966cfada337","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.061Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-829fc137343d6de4170c6e4d: done: 0.258452 sec","job_status":"done","queue_duration_s":0.49423,"scheduling_latency_s":0.494245,"redis_calls":564,"redis_duration_s":0.082021,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000428,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000899,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.080694,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.203623,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.258452,"completed_at":"2026-10-10T10:48:53.813Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000475,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.523858] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.814Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"f3c8c6f8faeaaf9a1a353e32","created_at":"2026-10-10T10:48:53.080Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"74cc98496590edf53c5920101ef4d37f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.093Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-f3c8c6f8faeaaf9a1a353e32: start","job_status":"start","queue_duration_s":0.721754,"scheduling_latency_s":0.721775} gitlab # [ 256.586608] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.878Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"f3c8c6f8faeaaf9a1a353e32","created_at":"2026-10-10T10:48:53.080Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"74cc98496590edf53c5920101ef4d37f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.093Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-f3c8c6f8faeaaf9a1a353e32: done: 0.063757 sec","job_status":"done","queue_duration_s":0.721754,"scheduling_latency_s":0.721775,"redis_calls":70,"redis_duration_s":0.011269,"redis_read_bytes":119,"redis_write_bytes":42580,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.001416,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000627,"redis_queues_read_bytes":4,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.008766,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00046,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.051188,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.063757,"completed_at":"2026-10-10T10:48:53.878Z","load_balancing_strategy":"primary","db_duration_s":0.001429,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.588353] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.879Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"6e10d24da0328bba29b9320e","created_at":"2026-10-10T10:48:53.103Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"616ce8da06c63edb6bb8b14363989c44","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.114Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-6e10d24da0328bba29b9320e: start","job_status":"start","queue_duration_s":0.765193,"scheduling_latency_s":0.765211} gitlab # [ 256.636322] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.899Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"6e10d24da0328bba29b9320e","created_at":"2026-10-10T10:48:53.103Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"616ce8da06c63edb6bb8b14363989c44","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.114Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-6e10d24da0328bba29b9320e: done: 0.020044 sec","job_status":"done","queue_duration_s":0.765193,"scheduling_latency_s":0.765211,"redis_calls":10,"redis_duration_s":0.002535,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000463,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001595,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000477,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000289422,"duration_s":0.020044,"completed_at":"2026-10-10T10:48:53.899Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004071,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.637763] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.901Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"e4ee1169bd270841d2cbb9ac","created_at":"2026-10-10T10:48:53.133Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"48efb63de12c630a35e2459c22fd8295","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.144Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-e4ee1169bd270841d2cbb9ac: start","job_status":"start","queue_duration_s":0.756708,"scheduling_latency_s":0.756722} gitlab # [ 256.649896] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.925Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"e4ee1169bd270841d2cbb9ac","created_at":"2026-10-10T10:48:53.133Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"48efb63de12c630a35e2459c22fd8295","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.144Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-e4ee1169bd270841d2cbb9ac: done: 0.024198 sec","job_status":"done","queue_duration_s":0.756708,"scheduling_latency_s":0.756722,"redis_calls":9,"redis_duration_s":0.00195,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000443,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001086,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000421,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008838,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.024198,"completed_at":"2026-10-10T10:48:53.925Z","load_balancing_strategy":"primary","db_duration_s":0.010385,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.709400] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.926Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"d97e840ac584cd811847ea09","created_at":"2026-10-10T10:48:53.168Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"55f2e482298b570930f5e0727ed73375","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.177Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-d97e840ac584cd811847ea09: start","job_status":"start","queue_duration_s":0.748949,"scheduling_latency_s":0.748963} gitlab # [ 256.710669] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.961Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"d97e840ac584cd811847ea09","created_at":"2026-10-10T10:48:53.168Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-6a6eb91f812f4c2a","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"55f2e482298b570930f5e0727ed73375","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.177Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-d97e840ac584cd811847ea09: done: 0.034919 sec","job_status":"done","queue_duration_s":0.748949,"scheduling_latency_s":0.748963,"redis_calls":13,"redis_duration_s":0.0029300000000000003,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000893,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001066,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000971,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015804,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000379936,"exclusive_lock_hold_duration_s":0.019461691,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.034919,"completed_at":"2026-10-10T10:48:53.961Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013529,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.766965] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.962Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b1d7b8afa2c93b49e30e4da3","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b1d7b8afa2c93b49e30e4da3: start","job_status":"start","queue_duration_s":0.776371,"scheduling_latency_s":0.776385} gitlab # [ 256.767646] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.983Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b1d7b8afa2c93b49e30e4da3","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b1d7b8afa2c93b49e30e4da3: done: 0.020769 sec","job_status":"done","queue_duration_s":0.776371,"scheduling_latency_s":0.776385,"redis_calls":13,"redis_duration_s":0.0027660000000000002,"redis_read_bytes":12,"redis_write_bytes":1300,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000414,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":289,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001041,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":395,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001311,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":616,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011648,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000293613,"duration_s":0.020769,"completed_at":"2026-10-10T10:48:53.983Z","load_balancing_strategy":"primary","db_duration_s":0.004062,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.790153] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.984Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f638c1374b6d607887638789","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f638c1374b6d607887638789: start","job_status":"start","queue_duration_s":0.798543,"scheduling_latency_s":0.798556} gitlab # [ 256.801443] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f638c1374b6d607887638789","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f638c1374b6d607887638789: done: 0.012797 sec","job_status":"done","queue_duration_s":0.798543,"scheduling_latency_s":0.798556,"redis_calls":10,"redis_duration_s":0.002141,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000781,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00136,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007285,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012797,"completed_at":"2026-10-10T10:48:53.997Z","load_balancing_strategy":"primary","db_duration_s":0.001684,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.842650] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:53.998Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d3a92230ad609cbfa0c35af9","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d3a92230ad609cbfa0c35af9: start","job_status":"start","queue_duration_s":0.812395,"scheduling_latency_s":0.812408} gitlab # [ 256.846987] nix-daemon[2826]: accepted connection from pid 3696, user nixbot gitlab # [ 256.847654] nix-daemon[2826]: accepted connection from pid 3657, user nixbot gitlab # [ 256.848571] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.023Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d3a92230ad609cbfa0c35af9","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d3a92230ad609cbfa0c35af9: done: 0.02472 sec","job_status":"done","queue_duration_s":0.812395,"scheduling_latency_s":0.812408,"redis_calls":10,"redis_duration_s":0.0023,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001095,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001205,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018837,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02472,"completed_at":"2026-10-10T10:48:54.023Z","load_balancing_strategy":"primary","db_duration_s":0.002054,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.902023] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.025Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e905fd4293037f56793e3870","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e905fd4293037f56793e3870: start","job_status":"start","queue_duration_s":0.838939,"scheduling_latency_s":0.83898} gitlab # [ 256.902772] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.049Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e905fd4293037f56793e3870","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e905fd4293037f56793e3870: done: 0.024477 sec","job_status":"done","queue_duration_s":0.838939,"scheduling_latency_s":0.83898,"redis_calls":10,"redis_duration_s":0.002187,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000855,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001332,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017574,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024477,"completed_at":"2026-10-10T10:48:54.049Z","load_balancing_strategy":"primary","db_duration_s":0.001671,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.955991] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.051Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"65d93af0163498e1c5cf6feb","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-65d93af0163498e1c5cf6feb: start","job_status":"start","queue_duration_s":0.865101,"scheduling_latency_s":0.865127} gitlab # [ 256.956699] gitaly[893]: time="2026-10-10T10:48:54.176Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW5KWASEKVEQTYKYY4XEW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:24.175" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:54.175" grpc.time_ms=0.557 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 256.957471] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:54 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/52b80e7504818909a1267a8f37adb6d0ad445c95 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 105 gitlab # [ 256.957970] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.067Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"65d93af0163498e1c5cf6feb","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-65d93af0163498e1c5cf6feb: done: 0.015896 sec","job_status":"done","queue_duration_s":0.865101,"scheduling_latency_s":0.865127,"redis_calls":10,"redis_duration_s":0.001648,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00059,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001058,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010259,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015896,"completed_at":"2026-10-10T10:48:54.067Z","load_balancing_strategy":"primary","db_duration_s":0.00138,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.022657] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.067Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"132b9b1d3870cf5a33277c46","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-132b9b1d3870cf5a33277c46: start","job_status":"start","queue_duration_s":0.881854,"scheduling_latency_s":0.88187} gitlab # [ 257.023162] gitaly[893]: time="2026-10-10T10:48:54.212Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=5 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPW5KWASEKVEQTYKYY4XEW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:49:04.141" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:54.202" grpc.time_ms=10.786 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/52b80e7504818909a1267a8f37adb6d0ad445c95/statuses gitlab # [ 257.023787] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.079Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"132b9b1d3870cf5a33277c46","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-132b9b1d3870cf5a33277c46: done: 0.011695 sec","job_status":"done","queue_duration_s":0.881854,"scheduling_latency_s":0.88187,"redis_calls":10,"redis_duration_s":0.001303,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0005,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000803,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007549,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011695,"completed_at":"2026-10-10T10:48:54.079Z","load_balancing_strategy":"primary","db_duration_s":0.001083,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.070971] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.080Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"be6f4da4f6d9a1be6c2d3140","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-be6f4da4f6d9a1be6c2d3140: start","job_status":"start","queue_duration_s":0.894396,"scheduling_latency_s":0.894409} gitlab # [ 257.072067] gitaly[893]: time="2026-10-10T10:48:54.287Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW5QAP8JYCTX1YYX8FH32 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:24.286" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:54.287" grpc.time_ms=0.499 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.072984] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:54 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/52b80e7504818909a1267a8f37adb6d0ad445c95 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 99 gitlab # [ 257.073721] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.093Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"be6f4da4f6d9a1be6c2d3140","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-be6f4da4f6d9a1be6c2d3140: done: 0.013398 sec","job_status":"done","queue_duration_s":0.894396,"scheduling_latency_s":0.894409,"redis_calls":10,"redis_duration_s":0.00148,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000552,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000928,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007906,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013398,"completed_at":"2026-10-10T10:48:54.093Z","load_balancing_strategy":"primary","db_duration_s":0.002588,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/52b80e7504818909a1267a8f37adb6d0ad445c95/statuses, in 0.08 seconds) gitlab # [ 257.119712] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.095Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9e6d33924ae8031ad1b4b186","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9e6d33924ae8031ad1b4b186: start","job_status":"start","queue_duration_s":0.909042,"scheduling_latency_s":0.909058} gitlab # [ 257.120511] gitaly[893]: time="2026-10-10T10:48:54.314Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPW5QAP8JYCTX1YYX8FH32 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:49:04.288" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:54.308" grpc.time_ms=5.642 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.126959] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.111Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9e6d33924ae8031ad1b4b186","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9e6d33924ae8031ad1b4b186: done: 0.016568 sec","job_status":"done","queue_duration_s":0.909042,"scheduling_latency_s":0.909058,"redis_calls":10,"redis_duration_s":0.001629,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000645,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000984,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010864,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016568,"completed_at":"2026-10-10T10:48:54.111Z","load_balancing_strategy":"primary","db_duration_s":0.003797,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.188971] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.112Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"dac77edf9fcf416538bbb6d7","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-dac77edf9fcf416538bbb6d7: start","job_status":"start","queue_duration_s":0.9267,"scheduling_latency_s":0.926719} gitlab # [ 257.189753] gitaly[893]: time="2026-10-10T10:48:54.394Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW5TPZ5B79440JYEYXTE8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:24.192" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:54.392" grpc.time_ms=1.727 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.190540] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:54 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/52b80e7504818909a1267a8f37adb6d0ad445c95/statuses HTTP/1.1" 200 1313 "" "curl/8.21.0" 50 gitlab # [ 257.190993] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.134Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"dac77edf9fcf416538bbb6d7","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-dac77edf9fcf416538bbb6d7: done: 0.021996 sec","job_status":"done","queue_duration_s":0.9267,"scheduling_latency_s":0.926719,"redis_calls":10,"redis_duration_s":0.001406,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000596,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00081,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009242,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021996,"completed_at":"2026-10-10T10:48:54.134Z","load_balancing_strategy":"primary","db_duration_s":0.017537,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.237499] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.135Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3be7e4e84255f8a53f1492f2","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3be7e4e84255f8a53f1492f2: start","job_status":"start","queue_duration_s":0.94982,"scheduling_latency_s":0.949839} gitlab # [ 257.238298] gitaly[893]: time="2026-10-10T10:48:54.426Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW5VRRM08G1RG6DY1G0CB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T10:49:24.425" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T10:48:54.426" grpc.time_ms=0.345 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.242710] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.152Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3be7e4e84255f8a53f1492f2","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3be7e4e84255f8a53f1492f2: done: 0.016501 sec","job_status":"done","queue_duration_s":0.94982,"scheduling_latency_s":0.949839,"redis_calls":10,"redis_duration_s":0.001418,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000868,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008908,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016501,"completed_at":"2026-10-10T10:48:54.152Z","load_balancing_strategy":"primary","db_duration_s":0.005539,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.282170] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.153Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"79eebc2d6e2db42e554079dd","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-79eebc2d6e2db42e554079dd: start","job_status":"start","queue_duration_s":0.967277,"scheduling_latency_s":0.967292} gitlab # [ 257.288237] gitaly[893]: time="2026-10-10T10:48:54.433Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=14 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M4JPW5TPZ5B79440JYEYXTE8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T10:49:04.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T10:48:54.416" grpc.time_ms=16.99 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.300721] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.166Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"79eebc2d6e2db42e554079dd","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-79eebc2d6e2db42e554079dd: done: 0.012817 sec","job_status":"done","queue_duration_s":0.967277,"scheduling_latency_s":0.967292,"redis_calls":10,"redis_duration_s":0.00145,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000888,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008587,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012817,"completed_at":"2026-10-10T10:48:54.166Z","load_balancing_strategy":"primary","db_duration_s":0.001035,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.351992] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.167Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4d007a51a4c3aee60b529210","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4d007a51a4c3aee60b529210: start","job_status":"start","queue_duration_s":0.981008,"scheduling_latency_s":0.981022} gitlab # [ 257.352827] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.182Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4d007a51a4c3aee60b529210","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4d007a51a4c3aee60b529210: done: 0.015376 sec","job_status":"done","queue_duration_s":0.981008,"scheduling_latency_s":0.981022,"redis_calls":10,"redis_duration_s":0.001712,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000607,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001105,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015376,"completed_at":"2026-10-10T10:48:54.182Z","load_balancing_strategy":"primary","db_duration_s":0.003611,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.381199] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.183Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4597564b75305d99b9455672","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4597564b75305d99b9455672: start","job_status":"start","queue_duration_s":0.997558,"scheduling_latency_s":0.997576} gitlab # [ 257.392073] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:54 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/52b80e7504818909a1267a8f37adb6d0ad445c95 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 270 gitlab # [ 257.394465] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.198Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4597564b75305d99b9455672","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4597564b75305d99b9455672: done: 0.01515 sec","job_status":"done","queue_duration_s":0.997558,"scheduling_latency_s":0.997576,"redis_calls":10,"redis_duration_s":0.001488,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000735,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000753,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009099,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01515,"completed_at":"2026-10-10T10:48:54.198Z","load_balancing_strategy":"primary","db_duration_s":0.001179,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.447185] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.200Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"86f394e36d27adbd92508e75","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-86f394e36d27adbd92508e75: start","job_status":"start","queue_duration_s":1.014116,"scheduling_latency_s":1.014132} gitlab # [ 257.447870] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.215Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"86f394e36d27adbd92508e75","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-86f394e36d27adbd92508e75: done: 0.015054 sec","job_status":"done","queue_duration_s":1.014116,"scheduling_latency_s":1.014132,"redis_calls":10,"redis_duration_s":0.001422,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000872,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008378,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015054,"completed_at":"2026-10-10T10:48:54.215Z","load_balancing_strategy":"primary","db_duration_s":0.003191,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.490721] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.216Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"adc7495d24956ef9d7618c70","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-adc7495d24956ef9d7618c70: start","job_status":"start","queue_duration_s":1.030241,"scheduling_latency_s":1.030255} gitlab # [ 257.492988] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.229Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"adc7495d24956ef9d7618c70","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-adc7495d24956ef9d7618c70: done: 0.013238 sec","job_status":"done","queue_duration_s":1.030241,"scheduling_latency_s":1.030255,"redis_calls":10,"redis_duration_s":0.001408,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000876,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009276,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013238,"completed_at":"2026-10-10T10:48:54.229Z","load_balancing_strategy":"primary","db_duration_s":0.001067,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.539769] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.230Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"29c3184f02c28811fc18afad","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-29c3184f02c28811fc18afad: start","job_status":"start","queue_duration_s":1.044704,"scheduling_latency_s":1.044721} gitlab # [ 257.540460] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.245Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"29c3184f02c28811fc18afad","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-29c3184f02c28811fc18afad: done: 0.014218 sec","job_status":"done","queue_duration_s":1.044704,"scheduling_latency_s":1.044721,"redis_calls":10,"redis_duration_s":0.00358,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002884,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000696,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008231,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014218,"completed_at":"2026-10-10T10:48:54.244Z","load_balancing_strategy":"primary","db_duration_s":0.00096,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.588751] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.246Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b67750d86dea23fd74e53d9c","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b67750d86dea23fd74e53d9c: start","job_status":"start","queue_duration_s":1.060095,"scheduling_latency_s":1.060109} gitlab # [ 257.589890] nixbot[2661]: {"ts": "2026-10-10T10:48:54.787Z", "level": "info", "logger": "nixbot.effects_run", "msg": "effects discovered", "build_id": 2, "sha": "52b80e7504818909a1267a8f37adb6d0ad445c95", "branch": "master", "pr": null, "allowed": true, "effects_started": false, "effects": []} gitlab # [ 257.590538] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:54 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 43 gitlab # [ 257.590950] gitaly[893]: time="2026-10-10T10:48:54.857Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JPW68Y9EBV6K9E9GZZRVDA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:54.850" grpc.time_ms=6.853 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 257.592159] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.260Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b67750d86dea23fd74e53d9c","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b67750d86dea23fd74e53d9c: done: 0.014164 sec","job_status":"done","queue_duration_s":1.060095,"scheduling_latency_s":1.060109,"redis_calls":10,"redis_duration_s":0.001474,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000595,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000879,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008087,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014164,"completed_at":"2026-10-10T10:48:54.260Z","load_balancing_strategy":"primary","db_duration_s":0.00334,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.633304] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.261Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"08ee857c9a67d9c5e09f8557","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-08ee857c9a67d9c5e09f8557: start","job_status":"start","queue_duration_s":1.075145,"scheduling_latency_s":1.075159} gitlab # [ 257.640577] gitaly[893]: time="2026-10-10T10:48:54.902Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW6AB5QQHEVBEB6XRF85E grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:54.894" pid=893 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 257.647679] gitlab-workhorse[827]: time="2026-10-10T10:48:54Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 257.649518] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:54 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 43 gitlab # [ 257.650143] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.273Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"08ee857c9a67d9c5e09f8557","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-08ee857c9a67d9c5e09f8557: done: 0.011911 sec","job_status":"done","queue_duration_s":1.075145,"scheduling_latency_s":1.075159,"redis_calls":10,"redis_duration_s":0.001301,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00076,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00777,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011911,"completed_at":"2026-10-10T10:48:54.273Z","load_balancing_strategy":"primary","db_duration_s":0.001015,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.685076] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.274Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e8d082976c33ff1a3e580c30","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e8d082976c33ff1a3e580c30: start","job_status":"start","queue_duration_s":1.087923,"scheduling_latency_s":1.087936} gitlab # [ 257.696092] gitaly[893]: time="2026-10-10T10:48:54.902Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW6AB5QQHEVBEB6XRF85E grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:54.894" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 257.708254] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:54 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 60 gitlab # [ 257.709875] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.285Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e8d082976c33ff1a3e580c30","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e8d082976c33ff1a3e580c30: done: 0.011888 sec","job_status":"done","queue_duration_s":1.087923,"scheduling_latency_s":1.087936,"redis_calls":10,"redis_duration_s":0.001308,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000792,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007784,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011888,"completed_at":"2026-10-10T10:48:54.285Z","load_balancing_strategy":"primary","db_duration_s":0.001007,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.774102] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.287Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"afb49fd40a8bc036bf18b6fd","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-afb49fd40a8bc036bf18b6fd: start","job_status":"start","queue_duration_s":1.101187,"scheduling_latency_s":1.101219} gitlab # [ 257.774886] gitaly[893]: time="2026-10-10T10:48:54.902Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW6AB5QQHEVBEB6XRF85E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:54.894" grpc.time_ms=7.729 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 257.775623] gitlab-workhorse[827]: time="2026-10-10T10:48:55Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 257.776054] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/10/10:10:48:55 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 44 gitlab # [ 257.776522] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.301Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"afb49fd40a8bc036bf18b6fd","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-afb49fd40a8bc036bf18b6fd: done: 0.014012 sec","job_status":"done","queue_duration_s":1.101187,"scheduling_latency_s":1.101219,"redis_calls":10,"redis_duration_s":0.001448,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000596,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000852,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008277,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014012,"completed_at":"2026-10-10T10:48:54.301Z","load_balancing_strategy":"primary","db_duration_s":0.001073,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.806072] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.302Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6ca4ffd216e76b334cb489ac","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6ca4ffd216e76b334cb489ac: start","job_status":"start","queue_duration_s":1.116229,"scheduling_latency_s":1.116244} gitlab # [ 257.817541] gitaly[893]: time="2026-10-10T10:48:54.983Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=28 command.spawn_token_fork_ms=24 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JPW6CAZDXB8GMJV7TMPJQ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:54.953" grpc.time_ms=29.938 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 257.855157] gitaly[893]: time="2026-10-10T10:48:55.029Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW6E98VWJ6YFG5G5PXM3T grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:55.019" pid=893 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc gitlab # [ 257.855942] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.320Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6ca4ffd216e76b334cb489ac","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6ca4ffd216e76b334cb489ac: done: 0.017737 sec","job_status":"done","queue_duration_s":1.116229,"scheduling_latency_s":1.116244,"redis_calls":10,"redis_duration_s":0.003933,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002726,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001207,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017737,"completed_at":"2026-10-10T10:48:54.320Z","load_balancing_strategy":"primary","db_duration_s":0.001811,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.872957] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.321Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4c42fdc6e492502789056b65","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4c42fdc6e492502789056b65: start","job_status":"start","queue_duration_s":1.13528,"scheduling_latency_s":1.135297} gitlab # [ 257.917867] gitaly[893]: time="2026-10-10T10:48:55.029Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW6E98VWJ6YFG5G5PXM3T grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:55.019" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 257.918711] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.333Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4c42fdc6e492502789056b65","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4c42fdc6e492502789056b65: done: 0.012473 sec","job_status":"done","queue_duration_s":1.13528,"scheduling_latency_s":1.135297,"redis_calls":10,"redis_duration_s":0.001519,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000735,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000784,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008587,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012473,"completed_at":"2026-10-10T10:48:54.333Z","load_balancing_strategy":"primary","db_duration_s":0.00101,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.967497] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.334Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d06e85742f7683efce7fe8bb","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d06e85742f7683efce7fe8bb: start","job_status":"start","queue_duration_s":1.148882,"scheduling_latency_s":1.148896} gitlab # [ 257.968329] gitaly[893]: time="2026-10-10T10:48:55.029Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JPW6E98VWJ6YFG5G5PXM3T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T10:48:55.019" grpc.time_ms=10.693 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 257.968954] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.346Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d06e85742f7683efce7fe8bb","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d06e85742f7683efce7fe8bb: done: 0.011812 sec","job_status":"done","queue_duration_s":1.148882,"scheduling_latency_s":1.148896,"redis_calls":10,"redis_duration_s":0.001342,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000818,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00819,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011812,"completed_at":"2026-10-10T10:48:54.346Z","load_balancing_strategy":"primary","db_duration_s":0.000919,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.021884] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.347Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3214583c742755c00d9cb0f7","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3214583c742755c00d9cb0f7: start","job_status":"start","queue_duration_s":1.161817,"scheduling_latency_s":1.161831} gitlab # [ 258.023113] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.362Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3214583c742755c00d9cb0f7","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3214583c742755c00d9cb0f7: done: 0.014258 sec","job_status":"done","queue_duration_s":1.161817,"scheduling_latency_s":1.161831,"redis_calls":10,"redis_duration_s":0.001477,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000933,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010192,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014258,"completed_at":"2026-10-10T10:48:54.362Z","load_balancing_strategy":"primary","db_duration_s":0.00151,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.037809] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.364Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8b1379817ff3dc283892e32e","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8b1379817ff3dc283892e32e: start","job_status":"start","queue_duration_s":1.177891,"scheduling_latency_s":1.177907} gitlab # [ 258.049820] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.376Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8b1379817ff3dc283892e32e","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8b1379817ff3dc283892e32e: done: 0.013016 sec","job_status":"done","queue_duration_s":1.177891,"scheduling_latency_s":1.177907,"redis_calls":10,"redis_duration_s":0.001371,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000585,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000786,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013016,"completed_at":"2026-10-10T10:48:54.376Z","load_balancing_strategy":"primary","db_duration_s":0.001934,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.105352] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.377Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"749d1dac9eecae4b76a54270","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-749d1dac9eecae4b76a54270: start","job_status":"start","queue_duration_s":1.191831,"scheduling_latency_s":1.191844} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/52b80e7504818909a1267a8f37adb6d0ad445c95/statuses gitlab # [ 258.106194] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.389Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"749d1dac9eecae4b76a54270","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-749d1dac9eecae4b76a54270: done: 0.011565 sec","job_status":"done","queue_duration_s":1.191831,"scheduling_latency_s":1.191844,"redis_calls":10,"redis_duration_s":0.00152,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000732,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000788,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00715,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011565,"completed_at":"2026-10-10T10:48:54.389Z","load_balancing_strategy":"primary","db_duration_s":0.001444,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.152239] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.390Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f5d90fc7566271f4c13f08b6","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f5d90fc7566271f4c13f08b6: start","job_status":"start","queue_duration_s":1.204313,"scheduling_latency_s":1.204326} gitlab # [ 258.152979] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.406Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f5d90fc7566271f4c13f08b6","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f5d90fc7566271f4c13f08b6: done: 0.015864 sec","job_status":"done","queue_duration_s":1.204313,"scheduling_latency_s":1.204326,"redis_calls":10,"redis_duration_s":0.001462,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000557,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000905,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009187,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015864,"completed_at":"2026-10-10T10:48:54.406Z","load_balancing_strategy":"primary","db_duration_s":0.003176,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.198745] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.407Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d429620eb84946d73fcfd75d","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d429620eb84946d73fcfd75d: start","job_status":"start","queue_duration_s":1.221189,"scheduling_latency_s":1.221204} gitlab # [ 258.199828] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.430Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d429620eb84946d73fcfd75d","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d429620eb84946d73fcfd75d: done: 0.022995 sec","job_status":"done","queue_duration_s":1.221189,"scheduling_latency_s":1.221204,"redis_calls":10,"redis_duration_s":0.008578,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.007666,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000912,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.004,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008107,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022995,"completed_at":"2026-10-10T10:48:54.430Z","load_balancing_strategy":"primary","db_duration_s":0.005179,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/52b80e7504818909a1267a8f37adb6d0ad445c95/statuses, in 0.12 seconds) gitlab # [ 258.246097] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.434Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7c6e2d7dbabf84aa8983aef9","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7c6e2d7dbabf84aa8983aef9: start","job_status":"start","queue_duration_s":1.248172,"scheduling_latency_s":1.248188} (finished: subtest: eval and build post commit statuses, in 5.75 seconds) (finished: run the VM test script, in 334.85 seconds) test script finished in 334.87s cleanup kill QemuMachine (pid 46) gitlab # [ 258.246737] sidekiq-cluster[2535]: {"severity":"INFO","time":"2026-10-10T10:48:54.451Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7c6e2d7dbabf84aa8983aef9","created_at":"2026-10-10T10:48:52.938Z","trace_propagation_headers":{"sentry-trace":"6f8201e5a5104a2aa84c2f69b40a9cdb-a91d430260c14896","baggage":"sentry-trace_id=6f8201e5a5104a2aa84c2f69b40a9cdb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"94d6944b7505493109403e62344012b2","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T10:48:53.186Z","job_size_bytes":2,"pid":2535,"sidekiq_tid":"m7xv","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7c6e2d7dbabf84aa8983aef9: done: 0.016928 sec","job_status":"done","queue_duration_s":1.248172,"scheduling_latency_s":1.248188,"redis_calls":10,"redis_duration_s":0.005371,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001234,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.004137,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_cqemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/gxzhl7aaiid7zp3y47jqqiq7zg5mqpwp-python3-3.14.6/bin/python3.14) gitlab # i (finished: cleanup, in 1.47 seconds) additionally exposed symbols: gitlab, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-10-10T10:48:00.397Z', 'push_events': True, 'tag_push_events': True, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] [] [] [] [] [] [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '52b80e7504818909a1267a8f37adb6d0ad445c95', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': None, 'error': None, 'created_at': '2026-10-10T10:48:48.136742Z', 'started_at': '2026-10-10T10:48:48.485287Z', 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': None}] [{'id': 5, 'sha': '52b80e7504818909a1267a8f37adb6d0ad445c95', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-10T10:48:48.460Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '52b80e7504818909a1267a8f37adb6d0ad445c95', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-10T10:48:48.460Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '52b80e7504818909a1267a8f37adb6d0ad445c95', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-10T10:48:48.460Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '52b80e7504818909a1267a8f37adb6d0ad445c95', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-10T10:48:48.460Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '52b80e7504818909a1267a8f37adb6d0ad445c95', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 5s (1 warning)', 'created_at': '2026-10-10T10:48:48.460Z', 'started_at': None, 'finished_at': '2026-10-10T10:48:54.227Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '52b80e7504818909a1267a8f37adb6d0ad445c95', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-10-10T10:48:54.331Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '52b80e7504818909a1267a8f37adb6d0ad445c95', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 5s (1 warning)', 'created_at': '2026-10-10T10:48:48.460Z', 'started_at': None, 'finished_at': '2026-10-10T10:48:54.227Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '52b80e7504818909a1267a8f37adb6d0ad445c95', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-10-10T10:48:54.331Z', 'started_at': None, 'finished_at': '2026-10-10T10:48:54.447Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]