tribuchet: building on jamie Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 3600 seconds run the VM test script start all VMs gitlab: starting vm gitlab: QEMU running (pid 46) gitlab # Disk image does not exist, creating the virtualisation disk image... gitlab # Formatting '/build/vm-state-gitlab/tmp.kA5qW2voOb', fmt=raw size=1073741824 gitlab # mke2fs 1.47.4 (6-Mar-2025) gitlab # Discarding device blocks: 0/262144 done gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes gitlab # Filesystem UUID: b7c4a4c7-0abc-4370-acda-a0c904f0dbab gitlab # Superblock backups stored on blocks: gitlab # 32768, 98304, 163840, 229376 gitlab # gitlab # Allocating group tables: 0/8 done gitlab # Writing inode tables: 0/8 done gitlab # Creating journal (8192 blocks): done gitlab # Writing superblocks and filesystem accounting information: 0/8 done gitlab # (finished: start all VMs, in 49.13 seconds) gitlab # Virtualisation disk image created. gitlab: waiting for unit setup-gitlab.service gitlab # Creating Nix store image... gitlab: waiting for the VM to finish booting gitlab # Created Nix store image. gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)... gitlab # gitlab # gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)... gitlab # gitlab # gitlab # Booting from ROM... gitlab # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 6 17:27:09 UTC 2026 gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0ny25zx3myyca19js117nxp1yzmhg6r1-nixos-system-gitlab-test/init regInfo=/nix/store/2kyhg4skm5c26qp3k8js66sfsnxx6zhi-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.000000] BIOS-provided physical RAM map: gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved gitlab # [ 0.000000] NX (Execute Disable) protection: active gitlab # [ 0.000000] APIC: Static calls initialized gitlab # [ 0.000000] SMBIOS 2.8 present. gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 gitlab # [ 0.000000] DMI: Memory slots populated: 1/1 gitlab # [ 0.000000] Hypervisor detected: KVM gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 gitlab # [ 0.000000] kvm-clock: using sched offset of 510324597 cycles gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns gitlab # [ 0.000004] tsc: Detected 2400.010 MHz processor gitlab # [ 0.000810] last_pfn = 0x1c0000 max_arch_pfn = 0x10000000000 gitlab # [ 0.000836] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs gitlab # [ 0.000838] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT gitlab # [ 0.000876] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.002729] found SMP MP-table at [mem 0x000f5470-0x000f547f] gitlab # [ 0.002739] Using GB pages for direct mapping gitlab # [ 0.002900] RAMDISK: [mem 0xbe31c000-0xbffcffff] gitlab # [ 0.002906] ACPI: Early table checksum verification disabled gitlab # [ 0.002909] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) gitlab # [ 0.002912] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002916] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002922] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002924] ACPI: FACS 0x00000000BFFE0000 000040 gitlab # [ 0.002926] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002928] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002929] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002931] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2] gitlab # [ 0.002932] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e] gitlab # [ 0.002932] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] gitlab # [ 0.002933] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482] gitlab # [ 0.002933] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba] gitlab # [ 0.002934] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2] gitlab # [ 0.003147] No NUMA configuration found gitlab # [ 0.003148] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff] gitlab # [ 0.003151] NODE_DATA(0) allocated [mem 0x1bfff8780-0x1bfffdcff] gitlab # [ 0.005421] Zone ranges: gitlab # [ 0.005422] DMA [mem 0x0000000000001000-0x0000000000ffffff] gitlab # [ 0.005423] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] gitlab # [ 0.005425] Normal [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.005426] Device empty gitlab # [ 0.005426] Movable zone start for each node gitlab # [ 0.005427] Early memory node ranges gitlab # [ 0.005427] node 0: [mem 0x0000000000001000-0x000000000009efff] gitlab # [ 0.005428] node 0: [mem 0x0000000000100000-0x00000000bffd7fff] gitlab # [ 0.005429] node 0: [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.005430] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff] gitlab # [ 0.005452] On node 0, zone DMA: 1 pages in unavailable ranges gitlab # [ 0.005715] On node 0, zone DMA: 97 pages in unavailable ranges gitlab # [ 0.113288] On node 0, zone Normal: 40 pages in unavailable ranges gitlab # [ 0.113710] ACPI: PM-Timer IO Port: 0x608 gitlab # [ 0.113725] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) gitlab # [ 0.113756] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 gitlab # [ 0.113759] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) gitlab # [ 0.113760] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) gitlab # [ 0.113762] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) gitlab # [ 0.113763] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) gitlab # [ 0.113763] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) gitlab # [ 0.113766] ACPI: Using ACPI (MADT) for SMP configuration information gitlab # [ 0.113767] ACPI: HPET id: 0x8086a201 base: 0xfed00000 gitlab # [ 0.113772] TSC deadline timer available gitlab # [ 0.113776] CPU topo: Max. logical packages: 1 gitlab # [ 0.113777] CPU topo: Max. logical dies: 1 gitlab # [ 0.113777] CPU topo: Max. dies per package: 1 gitlab # [ 0.113780] CPU topo: Max. threads per core: 1 gitlab # [ 0.113781] CPU topo: Num. cores per package: 4 gitlab # [ 0.113782] CPU topo: Num. threads per package: 4 gitlab # [ 0.113782] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs gitlab # [ 0.113805] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() gitlab # [ 0.113821] kvm-guest: KVM setup pv remote TLB flush gitlab # [ 0.113824] kvm-guest: setup PV sched yield gitlab # [ 0.113839] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] gitlab # [ 0.113840] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] gitlab # [ 0.113841] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff] gitlab # [ 0.113843] [mem 0xc0000000-0xfeffbfff] available for PCI devices gitlab # [ 0.113844] Booting paravirtualized kernel on KVM gitlab # [ 0.113848] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns gitlab # [ 0.118308] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 gitlab # [ 0.120773] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288 gitlab # [ 0.120822] kvm-guest: PV spinlocks enabled gitlab # [ 0.120824] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) gitlab # [ 0.120826] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0ny25zx3myyca19js117nxp1yzmhg6r1-nixos-system-gitlab-test/init regInfo=/nix/store/2kyhg4skm5c26qp3k8js66sfsnxx6zhi-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.120920] Unknown kernel command line parameters "regInfo=/nix/store/2kyhg4skm5c26qp3k8js66sfsnxx6zhi-closure-info/registration", will be passed to user space. gitlab # [ 0.120934] random: crng init done gitlab # [ 0.120935] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes gitlab # [ 0.129859] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) gitlab # [ 0.134119] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) gitlab # [ 0.134205] software IO TLB: area num 4. gitlab # [ 0.207733] Fallback order for Node 0: 0 gitlab # [ 0.207740] Built 1 zonelists, mobility grouping on. Total pages: 1572726 gitlab # [ 0.207742] Policy zone: Normal gitlab # [ 0.210588] mem auto-init: stack:all(zero), heap alloc:on, heap free:off gitlab # [ 0.220124] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 gitlab # [ 0.233563] allocated 12582912 bytes of page_ext gitlab # [ 0.243486] ftrace: allocating 48717 entries in 192 pages gitlab # [ 0.243487] ftrace: allocated 192 pages with 2 groups gitlab # [ 0.244613] Dynamic Preempt: lazy gitlab # [ 0.244783] rcu: Preemptible hierarchical RCU implementation. gitlab # [ 0.244783] rcu: RCU event tracing is enabled. gitlab # [ 0.244784] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. gitlab # [ 0.244786] Trampoline variant of Tasks RCU enabled. gitlab # [ 0.244786] Rude variant of Tasks RCU enabled. gitlab # [ 0.244786] Tracing variant of Tasks RCU enabled. gitlab # [ 0.244787] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. gitlab # [ 0.244788] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 gitlab # [ 0.244813] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.244815] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.244816] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.249146] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16 gitlab # [ 0.249438] rcu: srcu_init: Setting srcu_struct sizes based on contention. gitlab # [ 0.249447] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns gitlab # [ 0.249556] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) gitlab # [ 0.253101] Console: colour VGA+ 80x25 gitlab # [ 0.253104] printk: legacy console [tty0] enabled gitlab # [ 0.283363] printk: legacy console [ttyS0] enabled gitlab # [ 0.382175] ACPI: Core revision 20250807 gitlab # [ 0.383063] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns gitlab # [ 0.384555] APIC: Switch to symmetric I/O mode setup gitlab # [ 0.385539] x2apic enabled gitlab # [ 0.386312] APIC: Switched APIC routing to: physical x2apic gitlab # [ 0.387189] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask() gitlab # [ 0.388331] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself() gitlab # [ 0.389769] kvm-guest: setup PV IPIs gitlab # [ 0.391290] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 gitlab # [ 0.392361] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns gitlab # [ 0.394063] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400010) gitlab # [ 0.395146] x86/cpu: User Mode Instruction Prevention (UMIP) activated gitlab # [ 0.397183] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 gitlab # [ 0.399060] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 gitlab # [ 0.400064] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto gitlab # [ 0.402060] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl gitlab # [ 0.403061] Transient Scheduler Attacks: Vulnerable: No microcode gitlab # [ 0.404060] Spectre V2 : Mitigation: Enhanced / Automatic IBRS gitlab # [ 0.405060] Speculative Return Stack Overflow: Mitigation: Safe RET gitlab # [ 0.406059] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization gitlab # [ 0.407067] Spectre V2 : Enabling IBPB for BPF gitlab # [ 0.408061] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier gitlab # [ 0.409061] active return thunk: srso_alias_return_thunk gitlab # [ 0.410779] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' gitlab # [ 0.412060] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' gitlab # [ 0.413060] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' gitlab # [ 0.414060] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' gitlab # [ 0.415060] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' gitlab # [ 0.416061] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' gitlab # [ 0.417060] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' gitlab # [ 0.418061] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 gitlab # [ 0.419061] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 gitlab # [ 0.420060] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 gitlab # [ 0.421061] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 gitlab # [ 0.422060] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 gitlab # [ 0.423060] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format. gitlab # [ 0.453030] Freeing SMP alternatives memory: 44K gitlab # [ 0.453676] pid_max: default: 32768 minimum: 301 gitlab # [ 0.454769] LSM: initializing lsm=capability,landlock,yama,bpf,ima gitlab # [ 0.455946] landlock: Up and running. gitlab # [ 0.456537] Yama: becoming mindful. gitlab # [ 0.457272] LSM support for eBPF active gitlab # [ 0.458281] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.459204] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.462436] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) gitlab # [ 0.463678] Performance Events: Fam17h+ core perfctr, AMD PMU driver. gitlab # [ 0.464071] ... version: 2 gitlab # [ 0.464808] ... bit width: 48 gitlab # [ 0.465063] ... generic counters: 6 gitlab # [ 0.465834] ... generic bitmap: 000000000000003f gitlab # [ 0.466063] ... fixed-purpose counters: 0 gitlab # [ 0.466829] ... fixed-purpose bitmap: 0000000000000000 gitlab # [ 0.467063] ... value mask: 0000ffffffffffff gitlab # [ 0.468015] ... max period: 00007fffffffffff gitlab # [ 0.468821] ... global_ctrl mask: 000000000000003f gitlab # [ 0.469165] signal: max sigframe size: 3376 gitlab # [ 0.470000] rcu: Hierarchical SRCU implementation. gitlab # [ 0.470718] rcu: Max phase no-delay instances is 400. gitlab # [ 0.471254] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level gitlab # [ 0.476563] smp: Bringing up secondary CPUs ... gitlab # [ 0.477468] smpboot: x86: Booting SMP configuration: gitlab # [ 0.478071] .... node #0, CPUs: #1 #2 #3 gitlab # [ 0.481204] smp: Brought up 1 node, 4 CPUs gitlab # [ 0.482790] smpboot: Total of 4 processors activated (19200.08 BogoMIPS) gitlab # [ 0.483409] Memory: 6011596K/6290904K available (17210K kernel code, 2726K rwdata, 13576K rodata, 3640K init, 2992K bss, 260112K reserved, 0K cma-reserved) gitlab # [ 0.485063] devtmpfs: initialized gitlab # [ 0.486079] x86/mm: Memory block size: 128MB gitlab # [ 0.489244] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) gitlab # [ 0.490142] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). gitlab # [ 0.491157] pinctrl core: initialized pinctrl subsystem gitlab # [ 0.492367] PM: RTC time: 05:31:48, date: 2026-09-04 gitlab # [ 0.496011] NET: Registered PF_NETLINK/PF_ROUTE protocol family gitlab # [ 0.498176] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations gitlab # [ 0.499117] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations gitlab # [ 0.501423] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations gitlab # [ 0.502074] audit: initializing netlink subsys (disabled) gitlab # [ 0.503091] audit: type=2000 audit(1788499908.218:1): state=initialized audit_enabled=0 res=1 gitlab # [ 0.503327] thermal_sys: Registered thermal governor 'fair_share' gitlab # [ 0.505063] thermal_sys: Registered thermal governor 'bang_bang' gitlab # [ 0.506063] thermal_sys: Registered thermal governor 'step_wise' gitlab # [ 0.507063] thermal_sys: Registered thermal governor 'user_space' gitlab # [ 0.508062] thermal_sys: Registered thermal governor 'power_allocator' gitlab # [ 0.509078] cpuidle: using governor menu gitlab # [ 0.511275] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 gitlab # [ 0.512343] PCI: Using configuration type 1 for base access gitlab # [ 0.513064] PCI: Using configuration type 1 for extended access gitlab # [ 0.514251] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. gitlab # [ 0.517405] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages gitlab # [ 0.518064] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page gitlab # [ 0.519065] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages gitlab # [ 0.520063] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page gitlab # [ 0.525179] ACPI: Added _OSI(Module Device) gitlab # [ 0.527074] ACPI: Added _OSI(Processor Device) gitlab # [ 0.529069] ACPI: Added _OSI(Processor Aggregator Device) gitlab # [ 0.535964] ACPI: 1 ACPI AML tables successfully acquired and loaded gitlab # [ 0.543172] ACPI: Interpreter enabled gitlab # [ 0.543942] ACPI: PM: (supports S0 S3 S4 S5) gitlab # [ 0.544630] ACPI: Using IOAPIC for interrupt routing gitlab # [ 0.545085] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug gitlab # [ 0.546063] PCI: Using E820 reservations for host bridge windows gitlab # [ 0.547471] ACPI: Enabled 2 GPEs in block 00 to 0F gitlab # [ 0.560186] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) gitlab # [ 0.563083] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] gitlab # [ 0.567025] acpiphp: Slot [3] registered gitlab # [ 0.569179] acpiphp: Slot [4] registered gitlab # [ 0.571192] acpiphp: Slot [5] registered gitlab # [ 0.573149] acpiphp: Slot [6] registered gitlab # [ 0.574129] acpiphp: Slot [7] registered gitlab # [ 0.576154] acpiphp: Slot [8] registered gitlab # [ 0.578128] acpiphp: Slot [9] registered gitlab # [ 0.580126] acpiphp: Slot [10] registered gitlab # [ 0.581114] acpiphp: Slot [11] registered gitlab # [ 0.583135] acpiphp: Slot [12] registered gitlab # [ 0.584123] acpiphp: Slot [13] registered gitlab # [ 0.586105] acpiphp: Slot [14] registered gitlab # [ 0.587121] acpiphp: Slot [15] registered gitlab # [ 0.588106] acpiphp: Slot [16] registered gitlab # [ 0.590104] acpiphp: Slot [17] registered gitlab # [ 0.591104] acpiphp: Slot [18] registered gitlab # [ 0.592122] acpiphp: Slot [19] registered gitlab # [ 0.594106] acpiphp: Slot [20] registered gitlab # [ 0.595104] acpiphp: Slot [21] registered gitlab # [ 0.597104] acpiphp: Slot [22] registered gitlab # [ 0.598121] acpiphp: Slot [23] registered gitlab # [ 0.599084] acpiphp: Slot [24] registered gitlab # [ 0.599834] acpiphp: Slot [25] registered gitlab # [ 0.601083] acpiphp: Slot [26] registered gitlab # [ 0.601853] acpiphp: Slot [27] registered gitlab # [ 0.602085] acpiphp: Slot [28] registered gitlab # [ 0.602829] acpiphp: Slot [29] registered gitlab # [ 0.604091] acpiphp: Slot [30] registered gitlab # [ 0.604842] acpiphp: Slot [31] registered gitlab # [ 0.605090] PCI host bridge to bus 0000:00 gitlab # [ 0.605810] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] gitlab # [ 0.607064] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] gitlab # [ 0.608063] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] gitlab # [ 0.610071] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] gitlab # [ 0.611063] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 0.612064] pci_bus 0000:00: root bus resource [bus 00-ff] gitlab # [ 0.613199] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint gitlab # [ 0.615606] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint gitlab # [ 0.617529] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint gitlab # [ 0.620831] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f] gitlab # [ 0.622139] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk gitlab # [ 0.625069] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk gitlab # [ 0.627068] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk gitlab # [ 0.630069] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk gitlab # [ 0.632514] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint gitlab # [ 0.639156] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f] gitlab # [ 0.643852] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint gitlab # [ 0.648772] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI gitlab # [ 0.651084] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB gitlab # [ 0.654591] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint gitlab # [ 0.671151] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] gitlab # [ 0.673076] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] gitlab # [ 0.674073] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref] gitlab # [ 0.675386] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] gitlab # [ 0.676795] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.682126] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f] gitlab # [ 0.683077] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] gitlab # [ 0.684085] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref] gitlab # [ 0.715089] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] gitlab # [ 0.721873] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.732098] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f] gitlab # [ 0.734079] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] gitlab # [ 0.736110] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref] gitlab # [ 0.741557] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.748088] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf] gitlab # [ 0.749071] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] gitlab # [ 0.750088] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref] gitlab # [ 0.753639] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.758095] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df] gitlab # [ 0.759095] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] gitlab # [ 0.761086] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref] gitlab # [ 0.763326] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.767095] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f] gitlab # [ 0.768028] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] gitlab # [ 0.768893] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref] gitlab # [ 0.772317] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.775059] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff] gitlab # [ 0.777071] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] gitlab # [ 0.778109] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref] gitlab # [ 0.780828] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.784094] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff] gitlab # [ 0.784984] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] gitlab # [ 0.785845] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref] gitlab # [ 0.787069] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref] gitlab # [ 0.789283] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint gitlab # [ 0.791643] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] gitlab # [ 0.793094] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref] gitlab # [ 0.795315] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint gitlab # [ 0.800102] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f] gitlab # [ 0.801026] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] gitlab # [ 0.802874] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref] gitlab # [ 0.805298] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.808070] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f] gitlab # [ 0.808965] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff] gitlab # [ 0.809908] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref] gitlab # [ 0.816678] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 gitlab # [ 0.820459] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 gitlab # [ 0.822578] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 gitlab # [ 0.825521] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 gitlab # [ 0.828268] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 gitlab # [ 0.834271] iommu: Default domain type: Translated gitlab # [ 0.836068] iommu: DMA domain TLB invalidation policy: lazy mode gitlab # [ 0.840283] ACPI: bus type USB registered gitlab # [ 0.842222] usbcore: registered new interface driver usbfs gitlab # [ 0.844117] usbcore: registered new interface driver hub gitlab # [ 0.846095] usbcore: registered new device driver usb gitlab # [ 0.850941] NetLabel: Initializing gitlab # [ 0.853068] NetLabel: domain hash size = 128 gitlab # [ 0.854067] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO gitlab # [ 0.857163] NetLabel: unlabeled traffic allowed by default gitlab # [ 0.859089] PCI: Using ACPI for IRQ routing gitlab # [ 0.862120] pci 0000:00:02.0: vgaarb: setting as boot VGA device gitlab # [ 0.863059] pci 0000:00:02.0: vgaarb: bridge control possible gitlab # [ 0.863059] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none gitlab # [ 0.869074] vgaarb: loaded gitlab # [ 0.870445] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 gitlab # [ 0.872066] hpet0: 3 comparators, 64-bit 100.000000 MHz counter gitlab # [ 0.877233] clocksource: Switched to clocksource kvm-clock gitlab # [ 0.880600] VFS: Disk quotas dquot_6.6.0 gitlab # [ 0.881375] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) gitlab # [ 0.883070] pnp: PnP ACPI init gitlab # [ 0.884278] pnp: PnP ACPI: found 6 devices gitlab # [ 0.893465] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns gitlab # [ 0.895076] clocksource: Switched to clocksource acpi_pm gitlab # [ 0.896125] NET: Registered PF_INET protocol family gitlab # [ 0.898370] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) gitlab # [ 0.916780] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) gitlab # [ 0.918346] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) gitlab # [ 0.919829] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) gitlab # [ 0.923562] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) gitlab # [ 0.924972] TCP: Hash tables configured (established 65536 bind 65536) gitlab # [ 0.926172] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) gitlab # [ 0.927521] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.929000] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.930344] NET: Registered PF_UNIX/PF_LOCAL protocol family gitlab # [ 0.931386] NET: Registered PF_XDP protocol family gitlab # [ 0.932254] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] gitlab # [ 0.933339] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] gitlab # [ 0.934413] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] gitlab # [ 0.935578] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] gitlab # [ 0.936751] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 0.938113] pci 0000:00:01.0: PIIX3: Enabling Passive Release gitlab # [ 0.939113] pci 0000:00:00.0: Limiting direct PCI/PCI transfers gitlab # [ 0.941473] ACPI: \_SB_.LNKD: Enabled at IRQ 11 gitlab # [ 0.944029] PCI: CLS 0 bytes, default 64 gitlab # [ 0.944859] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) gitlab # [ 0.945392] Trying to unpack rootfs image as initramfs... gitlab # [ 0.945968] software IO TLB: mapped [mem 0x00000000ba31c000-0x00000000be31c000] (64MB) gitlab # [ 0.946087] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns gitlab # [ 0.969478] Initialise system trusted keyrings gitlab # [ 0.970890] workingset: timestamp_bits=40 max_order=21 bucket_order=0 gitlab # [ 0.982363] Key type asymmetric registered gitlab # [ 0.983107] Asymmetric key parser 'x509' registered gitlab # [ 0.984043] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) gitlab # [ 0.985914] io scheduler mq-deadline registered gitlab # [ 0.986787] io scheduler kyber registered gitlab # [ 0.990341] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled gitlab # [ 0.991699] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A gitlab # [ 0.994112] Linux agpgart interface v0.103 gitlab # [ 0.994951] ACPI: bus type drm_connector registered gitlab # [ 0.997467] usbcore: registered new interface driver usbserial_generic gitlab # [ 0.998663] usbserial: USB Serial support registered for generic gitlab # [ 0.999750] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. gitlab # [ 0.999750] Please enable it if your BIOS has the CPPC option. gitlab # [ 1.002109] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled gitlab # [ 1.003490] drop_monitor: Initializing network drop monitor service gitlab # [ 1.004765] NET: Registered PF_INET6 protocol family gitlab # [ 1.006504] Segment Routing with IPv6 gitlab # [ 1.007208] In-situ OAM (IOAM) with IPv6 gitlab # [ 1.009247] IPI shorthand broadcast: enabled gitlab # [ 1.013698] sched_clock: Marking stable (877011345, 136613247)->(1039901987, -26277395) gitlab # [ 1.015643] registered taskstats version 1 gitlab # [ 1.016699] Loading compiled-in X.509 certificates gitlab # [ 1.026411] Demotion targets for Node 0: null gitlab # [ 1.027590] Key type .fscrypt registered gitlab # [ 1.028307] Key type fscrypt-provisioning registered gitlab # [ 1.029298] ima: No TPM chip found, activating TPM-bypass! gitlab # [ 1.030242] ima: Allocated hash algorithm: sha1 gitlab # [ 1.031104] ima: No architecture policies found gitlab # [ 1.032179] PM: Magic number: 10:324:516 gitlab # [ 1.033804] RAS: Correctable Errors collector initialized. gitlab # [ 1.038236] clk: Disabling unused clocks gitlab # [ 1.038982] PM: genpd: Disabling unused power domains gitlab # [ 1.231931] Freeing initrd memory: 29392K gitlab # [ 1.235687] Freeing unused decrypted memory: 2028K gitlab # [ 1.238409] Freeing unused kernel image (initmem) memory: 3640K gitlab # [ 1.239523] Write protecting the kernel read-only data: 32768k gitlab # [ 1.241721] Freeing unused kernel image (text/rodata gap) memory: 1220K gitlab # [ 1.243378] Freeing unused kernel image (rodata/data gap) memory: 760K gitlab # [ 1.294427] x86/mm: Checked W+X mappings: passed, no W+X pages found. gitlab # [ 1.295538] Run /init as init process gitlab # [ 1.308654] systemd[1]: Inserted module 'autofs4' gitlab # [ 1.331670] fuse: init (API version 7.45) gitlab # [ 1.339837] ACPI: \_SB_.LNKC: Enabled at IRQ 10 gitlab # [ 1.346064] ACPI: \_SB_.LNKA: Enabled at IRQ 10 gitlab # [ 1.349474] ACPI: \_SB_.LNKB: Enabled at IRQ 11 gitlab # [ 1.385809] systemd[1]: Successfully made /usr/ read-only. gitlab # [ 1.722111] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 1.764551] systemd[1]: Detected virtualization kvm. gitlab # [ 1.766791] systemd[1]: Detected architecture x86-64. gitlab # [ 1.768979] systemd[1]: Running in initrd. gitlab # [ 1.771463] systemd[1]: Initializing machine ID from random generator. gitlab # [ 1.774100] systemd[1]: Hostname set to . gitlab # [ 1.886920] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 1.955237] systemd[1]: Queued start job for default target Initrd Default Target. gitlab # [ 1.964329] systemd[1]: Created slice Slice /system/modprobe. gitlab # [ 1.965691] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 1.967540] systemd[1]: Expecting device /dev/disk/by-label/nix-store... gitlab # [ 1.969946] systemd[1]: Expecting device /dev/disk/by-label/nixos... gitlab # [ 1.971997] systemd[1]: Reached target Path Units. gitlab # [ 1.973000] systemd[1]: Reached target Slice Units. gitlab # [ 1.973983] systemd[1]: Reached target Swaps. gitlab # [ 1.974900] systemd[1]: Reached target Timer Units. gitlab # [ 1.975964] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 1.977269] systemd[1]: Listening on Journal Socket (/dev/log). gitlab # [ 1.978468] systemd[1]: Listening on Journal Sockets. gitlab # [ 1.979561] systemd[1]: Listening on udev Control Socket. gitlab # [ 1.980730] systemd[1]: Listening on udev Kernel Socket. gitlab # [ 1.981773] systemd[1]: Reached target Socket Units. gitlab # [ 1.984082] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 1.986413] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 1.988829] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 1.999695] systemd[1]: Starting Journal Service... gitlab # [ 2.003715] systemd[1]: Starting Load Kernel Modules... gitlab # [ 2.004768] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 2.011715] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 2.019753] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 2.021725] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 2.023337] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 2.024915] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config gitlab # [ 2.028034] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 2.049579] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 2.053490] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 2.053732] systemd-journald[92]: Collecting audit messages is disabled. gitlab # [ 2.057861] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. gitlab # [ 2.060219] netfs: FS-Cache loaded gitlab # [ 2.066643] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev gitlab # [ 2.070220] systemd[1]: Finished Load Kernel Modules. gitlab # [ 2.072767] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 2.075773] 9pnet: Installing 9P2000 support gitlab # [ 2.085477] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 2.087097] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 2.088830] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 2.090311] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 2.091699] systemd[1]: Reached target Local File Systems. gitlab # [ 2.093792] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 1.958592] systemd-modules-l[ 2.095379] systemd[1]: Started Journal Service. gitlab # oad[93]: Using 4 probe threads gitlab # [ 1.960627] systemd-modules-load[93]: Inserted module 'virtio_balloon' gitlab # [ 1.961776] systemd-modules-load[93]: Inserted module 'virtio_gpu' gitlab # [ 1.962818] systemd-modules-load[93]: Inserted module 'dm_mod' gitlab # [ 1.963762] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 1.964721] systemd[1]: Starting Create System Files and Directories... gitlab # [ 1.983932] systemd[1]: Finished Create System Files and Directories. gitlab # [ 1.993813] systemd-udevd[106]: Using default interface naming scheme 'v261'. gitlab # [ 2.006293] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 2.034161] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 2.035134] systemd[1]: Reached target System Initialization. gitlab # [ 2.036036] systemd[1]: Reached target Basic System. gitlab # [ 2.253472] SCSI subsystem initialized gitlab # [ 2.258407] virtio_blk virtio4: 4/0/0 default/read/poll queues gitlab # [ 2.263009] uhci_hcd 0000:00:01.2: UHCI Host Controller gitlab # [ 2.266958] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) gitlab # [ 2.268188] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1 gitlab # [ 2.269296] uhci_hcd 0000:00:01.2: detected 2 ports gitlab # [ 2.270160] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140 gitlab # [ 2.278069] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 gitlab # [ 2.279402] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 gitlab # [ 2.280469] usb usb1: Product: UHCI Host Controller gitlab # [ 2.281285] usb usb1: Manufacturer: Linux 6.18.43 uhci_hcd gitlab # [ 2.282155] usb usb1: SerialNumber: 0000:00:01.2 gitlab # [ 2.286728] virtio_blk virtio5: 4/0/0 default/read/poll queues gitlab # [ 2.292307] hub 1-0:1.0: USB hub found gitlab # [ 2.292383] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 gitlab # [ 2.292522] hub 1-0:1.0: 2 ports detected gitlab # [ 2.292677] scsi host0: ata_piix gitlab # [ 2.296077] serio: i8042 KBD port at 0x60,0x64 irq 1 gitlab # [ 2.296841] serio: i8042 AUX port at 0x60,0x64 irq 12 gitlab # [ 2.297871] scsi host1: ata_piix gitlab # [ 2.298593] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0 gitlab # [ 2.299902] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0 gitlab # [ 2.336452] virtio_blk virtio5: [vdb] 11842888 512-byte logical blocks (6.06 GB/5.65 GiB) gitlab # [ 2.208760] systemd[1]: Starting Virtual Console Setup... gitlab # [ 2.233620] systemd-vconsole-setup[154]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 2.235362] systemd[1]: Finished Virtual Console Setup. gitlab # [ 2.459033] ata2: found unknown device (class 0) gitlab # [ 2.462206] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 gitlab # [ 2.466832] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 gitlab # [ 2.358528] (udev-worker)[128]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 2.365440] (udev-worker)[128]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.370340] (udev-worker)[131]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.515960] usb 1-1: new full-speed USB device number 2 using uhci_hcd gitlab # [ 2.516401] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 gitlab # [ 2.396942] systemd[1]: Found device /dev/disk/by-label/nixos. gitlab # [ 2.401461] systemd[1]: Found device /dev/disk/by-label/nix-store. gitlab # [ 2.403409] systemd[1]: Reached target Initrd Root Device. gitlab # [ 2.405107] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... gitlab # [ 2.558803] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray gitlab # [ 2.454923] systemd-fsck[161]: nixos: clean, 12/65536 files, 13019/262144 blocks gitlab # [ 2.597411] cdrom: Uniform CD-ROM driver Revision: 3.20 gitlab # [ 2.463385] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. gitlab # [ 2.685807] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 gitlab # [ 2.689227] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 gitlab # [ 2.692015] usb 1-1: Product: QEMU USB Tablet gitlab # [ 2.693682] usb 1-1: Manufacturer: QEMU gitlab # [ 2.695230] usb 1-1: SerialNumber: 28754-0000:00:01.2-1 gitlab # [ 2.742777] hid: raw HID events driver (C) Jiri Kosina gitlab # [ 2.754647] usbcore: registered new interface driver usbhid gitlab # [ 2.756433] usbhid: USB HID core driver gitlab # [ 2.764145] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 gitlab # [ 2.768449] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0 gitlab # [ 2.886626] systemd[1]: Mounting /sysroot... gitlab # [ 3.181202] EXT4-fs (vda): mounted filesystem b7c4a4c7-0abc-4370-acda-a0c904f0dbab r/w with ordered data mode. Quota mode: none. gitlab # [ 3.051074] systemd[1]: Mounted /sysroot. gitlab # [ 3.052975] systemd[1]: Reached target Initrd Root File System. gitlab # [ 3.055254] systemd[1]: Mounting /sysroot/nix/.ro-store... gitlab # [ 3.057384] systemd[1]: Mounting /sysroot/nix/.rw-store... gitlab # [ 3.059455] systemd[1]: Mounting /sysroot/run... gitlab # [ 3.061321] systemd[1]: Mounting /sysroot/tmp/shared... gitlab # [ 3.074297] systemd[1]: Mounting /sysroot/tmp/xchg... gitlab # [ 3.076660] systemd[1]: Starting Mountpoints Configured in the Real Root... gitlab # [ 3.086558] systemd-sysroot-fstab-check[180]: /sysroot should be mounted in the initrd, will request daemon-reload. gitlab # [ 3.090204] systemd[1]: Reload requested from client PID 180 ('systemd-sysroot') (unit initrd-parse-etc.service)... gitlab # [ 3.093409] systemd[1]: Reloading... gitlab # [ 3.239015] 9p: Installing v9fs 9p2000 file system support gitlab # [ 3.250481] erofs (device vdb): mounted with root inode @ nid 128. gitlab # [ 3.161375] systemd[1]: Reloading finished in 69 ms. gitlab # [ 3.185540] systemd-sysroot-fstab-check[180]: Requesting initrd-fs.target/start/replace... gitlab # [ 3.188471] systemd[1]: Mounted /sysroot/nix/.ro-store. gitlab # [ 3.190380] systemd[1]: Mounted /sysroot/nix/.rw-store. gitlab # [ 3.192237] systemd[1]: Mounted /sysroot/run. gitlab # [ 3.193835] systemd[1]: Mounted /sysroot/tmp/shared. gitlab # [ 3.195568] systemd[1]: Mounted /sysroot/tmp/xchg. gitlab # [ 3.197327] systemd-sysroot-fstab-check[180]: Requesting swap.target/start/replace... gitlab # [ 3.200259] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.202718] systemd[1]: initrd-parse-etc.service: Deactivated successfully. gitlab # [ 3.205263] systemd[1]: Finished Mountpoints Configured in the Real Root. gitlab # [ 3.207636] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. gitlab # [ 3.228479] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.231068] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.887085] systemd[1]: Mounting /sysroot/nix/store... gitlab # [ 3.938973] systemd[1]: Mounted /sysroot/nix/store. gitlab # [ 3.941079] systemd[1]: Reached target Initrd File Systems. gitlab # [ 3.943175] systemd[1]: Starting Find NixOS closure... gitlab # [ 3.945096] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... gitlab # [ 3.971306] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. gitlab # [ 3.974234] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. gitlab # [ 3.989888] systemd[1]: Finished Find NixOS closure. gitlab # [ 3.991810] systemd[1]: Reached target Initrd Default Target. gitlab # [ 3.993886] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... gitlab # [ 4.025517] systemd[1]: Stopped target Initrd Default Target. gitlab # [ 4.027694] systemd[1]: Stopped target Basic System. gitlab # [ 4.029590] systemd[1]: Stopped target Initrd Root Device. gitlab # [ 4.031609] systemd[1]: Stopped target Path Units. gitlab # [ 4.033429] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. gitlab # [ 4.036178] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. gitlab # [ 4.042259] systemd[1]: Stopped target Slice Units. gitlab # [ 4.044341] systemd[1]: Stopped target Socket Units. gitlab # [ 4.046516] systemd[1]: Stopped target System Initialization. gitlab # [ 4.049577] systemd[1]: Stopped target Swaps. gitlab # [ 4.051823] systemd[1]: Stopped target Timer Units. gitlab # [ 4.053709] systemd[1]: dbus.socket: Deactivated successfully. gitlab # [ 4.056064] systemd[1]: Closed D-Bus System Message Bus Socket. gitlab # [ 4.058227] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. gitlab # [ 4.060722] systemd[1]: Stopped Find NixOS closure. gitlab # [ 4.062561] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 4.065680] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 4.067395] systemd[1]: systemd-sysctl.service: Deactivated successfully. gitlab # [ 4.069219] systemd[1]: Stopped Apply Kernel Variables. gitlab # [ 4.070692] systemd[1]: systemd-modules-load.service: Deactivated successfully. gitlab # [ 4.072662] systemd[1]: Stopped Load Kernel Modules. gitlab # [ 4.074141] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. gitlab # [ 4.076328] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. gitlab # [ 4.078398] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. gitlab # [ 4.080246] systemd[1]: Stopped Create System Files and Directories. gitlab # [ 4.081158] systemd[1]: Stopped target Local File Systems. gitlab # [ 4.081939] systemd[1]: Stopped target Preparation for Local File Systems. gitlab # [ 4.082962] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. gitlab # [ 4.083985] systemd[1]: Stopped Coldplug All udev Devices. gitlab # [ 4.084742] systemd[1]: Stopping Rule-based Manager for Device Events and Files... gitlab # [ 4.085799] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 4.086987] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 4.087785] systemd[1]: initrd-cleanup.service: Deactivated successfully. gitlab # [ 4.088770] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. gitlab # [ 4.089702] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 4.090685] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 4.091549] systemd[1]: systemd-udevd.service: Deactivated successfully. gitlab # [ 4.092529] systemd[1]: Stopped Rule-based Manager for Device Events and Files. gitlab # [ 4.093524] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. gitlab # [ 4.094584] systemd[1]: Closed udev Control Socket. gitlab # [ 4.095334] systemd[1]: Starting Cleanup udev Database... gitlab # [ 4.096117] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. gitlab # [ 4.097218] systemd[1]: Stopped Create Static Device Nodes in /dev. gitlab # [ 4.098122] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. gitlab # [ 4.099303] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. gitlab # [ 4.100351] systemd[1]: kmod-static-nodes.service: Deactivated successfully. gitlab # [ 4.101322] systemd[1]: Stopped Create List of Static Device Nodes. gitlab # [ 4.107891] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. gitlab # [ 4.109402] systemd[1]: Finished Cleanup udev Database. gitlab # [ 4.110724] systemd[1]: Reached target Switch Root. gitlab # [ 4.112759] systemd[1]: Starting NixOS Activation... gitlab # [ 4.258520] initrd-nixos-activation-start[232]: booting system configuration /nix/store/0ny25zx3myyca19js117nxp1yzmhg6r1-nixos-system-gitlab-test gitlab # [ 4.308152] initrd-nixos-activation-start[232]: running activation script... gitlab # [ 4.582605] initrd-nixos-activation-start[255]: setting up /etc... gitlab # [ 4.686619] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. gitlab # [ 4.688251] systemd[1]: Finished NixOS Activation. gitlab # [ 4.689951] systemd[1]: Starting Switch Root... gitlab # [ 4.712069] systemd[1]: Switching root. gitlab # [ 4.912976] systemd-journald[92]: Received SIGTERM from PID 1 (systemd). gitlab # [ 5.045025] NET: Registered PF_VSOCK protocol family gitlab # [ 5.425998] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 5.436205] systemd[1]: Detected virtualization kvm. gitlab # [ 5.438104] systemd[1]: Detected architecture x86-64. gitlab # [ 5.440056] systemd[1]: Detected first boot. gitlab # [ 5.442513] systemd[1]: Initializing machine ID from random generator. gitlab # [ 5.585203] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 5.681345] systemd[1]: Applying preset policy. gitlab # [ 5.815183] systemd[1]: Populated /etc with preset unit settings. gitlab # [ 5.934636] systemd[1]: initrd-switch-root.service: Deactivated successfully. gitlab # [ 5.936429] systemd[1]: Stopped initrd-switch-root.service. gitlab # [ 5.938889] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. gitlab # [ 5.940966] systemd[1]: Created slice Slice /system/getty. gitlab # [ 5.942342] systemd[1]: Created slice GitLab DevOps Platform Slice. gitlab # [ 5.943904] systemd[1]: Created slice User and Session Slice. gitlab # [ 5.944923] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 5.946147] systemd[1]: Started Forward Password Requests to Wall Directory Watch. gitlab # [ 5.947252] systemd[1]: Expecting device /dev/hvc0... gitlab # [ 5.947994] systemd[1]: Expecting device /dev/ttyS0... gitlab # [ 5.948822] systemd[1]: Reached target Local Encrypted Volumes. gitlab # [ 5.949695] systemd[1]: Stopped target initrd-fs.target. gitlab # [ 5.950463] systemd[1]: Stopped target initrd-root-fs.target. gitlab # [ 5.951301] systemd[1]: Stopped target initrd-switch-root.target. gitlab # [ 5.952186] systemd[1]: Reached target Virtual Machines and Containers. gitlab # [ 5.953169] systemd[1]: Reached target Path Units. gitlab # [ 5.953901] systemd[1]: Reached target Remote File Systems. gitlab # [ 5.954817] systemd[1]: Reached target Slice Units. gitlab # [ 5.955565] systemd[1]: Reached target Swaps. gitlab # [ 5.957027] systemd[1]: Listening on Query the User Interactively for a Password. gitlab # [ 5.958993] systemd[1]: Listening on Process Core Dump Socket. gitlab # [ 5.960383] systemd[1]: Listening on Credential Encryption/Decryption. gitlab # [ 5.961999] systemd[1]: Listening on Factory Reset Management. gitlab # [ 5.963020] systemd[1]: Listening on Hostname Service Socket. gitlab # [ 5.965383] systemd[1]: Starting Journal Log Access Socket... gitlab # [ 5.966839] systemd[1]: Listening on Journal Audit Socket. gitlab # [ 5.968843] systemd[1]: Listening on Console Output Muting Service Socket. gitlab # [ 5.970042] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. gitlab # [ 5.971211] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.972630] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 5.975495] systemd[1]: Listening on Disk Repartitioning Service Socket. gitlab # [ 5.976543] systemd[1]: Listening on udev Control Socket. gitlab # [ 5.977480] systemd[1]: Listening on udev Varlink Socket. gitlab # [ 5.980499] systemd[1]: Mounting Huge Pages File System... gitlab # [ 5.982514] systemd[1]: Mounting POSIX Message Queue File System... gitlab # [ 5.984543] systemd[1]: Mounting Kernel Debug File System... gitlab # [ 5.987694] systemd[1]: Mounting Kernel Trace File System... gitlab # [ 6.000936] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 6.002056] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 6.006784] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 6.009400] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm gitlab # [ 6.042370] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 6.051256] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 6.060192] systemd[1]: Mounting FUSE Control File System... gitlab # [ 6.063279] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 6.075201] systemd[1]: Starting Journal Service... gitlab # [ 6.078777] systemd[1]: Starting Load Kernel Modules... gitlab # [ 6.083819] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... gitlab # [ 6.088768] systemd[1]: Starting Remount Root and Kernel File Systems... gitlab # [ 6.090240] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.096694] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 6.099839] systemd[1]: Listening on Journal Log Access Socket. gitlab # [ 6.100518] loop: module loaded gitlab # [ 6.100898] systemd[1]: Mounted Huge Pages File System. gitlab # [ 6.102244] systemd[1]: Mounted POSIX Message Queue File System. gitlab # [ 6.103268] systemd[1]: Mounted Kernel Debug File System. gitlab # [ 6.104214] systemd[1]: Mounted Kernel Trace File System. gitlab # [ 6.106705] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 6.108081] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 6.112188] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 6.112508] systemd-journald[325]: Collecting audit messages is enabled. gitlab # [ 6.113330] systemd[1]: Mounted FUSE Control File System. gitlab # [ 6.116580] EXT4-fs (vda): re-mounted b7c4a4c7-0abc-4370-acda-a0c904f0dbab. gitlab # [ 6.118531] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 6.123686] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 5.996576] systemd[1]: Queued start job for default target Multi-User System. gitlab # [ 6.133093] systemd[1]: Finished Load Kernel Modules. gitlab # [ 5.997815] systemd[1]: Unnecessary job was removed for /dev/vdb. gitlab # [ 6.136455] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. gitlab # [ 6.001224] systemd[1]: systemd-journald.service: Deactivated successfully. gitlab # [ 6.002516] systemd-modules-l[ 6.138969] systemd[1]: Started Journal Service. gitlab # oad[326]: Using 3 probe threads gitlab # [ 6.004247] systemd-modules-load[326]: Inserted module 'loop' gitlab # [ 6.005324] systemd-modules-load[326]: Inserted module 'tls' gitlab # [ 6.006788] systemd-oomd[327]: No swap; memory pressure usage will be degraded gitlab # [ 6.008495] systemd[1]: Finished Remount Root and Kernel File Systems. gitlab # [ 6.009681] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 6.010749] systemd[1]: Listening on Disk Image Download Service Socket. gitlab # [ 6.011862] systemd[1]: Starting Firewall... gitlab # [ 6.012621] systemd[1]: Starting Flush Journal to Persistent Storage... gitlab # [ 6.013658] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 6.017340] systemd[1]: Starting Load/Save OS Random Seed... gitlab # [ 6.020105] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 6.021856] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.023392] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 6.163564] systemd-journald[325]: Received client request to flush runtime journal. gitlab # [ 6.073374] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 6.074522] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 6.075375] systemd[1]: Finished Load/Save OS Random Seed. gitlab # [ 6.076337] systemd[1]: Reached target First Boot Complete. gitlab # [ 6.077477] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 6.078295] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 6.079359] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 6.080552] systemd[1]: Finished Flush Journal to Persistent Storage. gitlab # [ 6.083435] systemd-udevd[360]: Using default interface naming scheme 'v261'. gitlab # [ 6.114742] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 6.136124] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 6.180938] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 6.201490] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. gitlab # [ 6.207203] (udev-worker)[444]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 6.209512] (udev-worker)[444]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.211354] (udev-worker)[437]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.219294] systemd[1]: Condition check resulted in Virtio network device being skipped. gitlab # [ 6.222143] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 6.223679] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 6.226117] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 6.228105] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.229550] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.241073] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. gitlab # [ 6.255172] systemd[1]: Finished Firewall. gitlab # [ 6.422573] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 gitlab # [ 6.426865] mousedev: PS/2 mouse device common for all mice gitlab # [ 6.429987] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console gitlab # [ 6.431061] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 gitlab # [ 6.435581] ACPI: button: Power Button [PWRF] gitlab # [ 6.441000] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD gitlab # [ 6.443896] Console: switching to colour dummy device 80x25 gitlab # [ 6.445886] [drm] Found bochs VGA, ID 0xb0c5. gitlab # [ 6.445888] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. gitlab # [ 6.448340] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic gitlab # [ 6.449197] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0 gitlab # [ 6.458919] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input4 gitlab # [ 6.470190] Console: switching to colour frame buffer device 160x50 gitlab # [ 6.472920] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device gitlab # [ 6.477571] rtc_cmos 00:05: RTC can wake from S4 gitlab # [ 6.480545] Floppy drive(s): fd0 is 2.88M AMI BIOS gitlab # [ 6.485169] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6 gitlab # [ 6.486620] rtc_cmos 00:05: registered as rtc0 gitlab # [ 6.487198] rtc_cmos 00:05: setting system clock to 2026-09-04T05:31:54 UTC (1788499914) gitlab # [ 6.488655] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 gitlab # [ 6.491961] FDC 0 is a S82078B gitlab # [ 6.492380] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs gitlab # [ 6.493009] parport_pc 00:03: reported by Plug and Play ACPI gitlab # [ 6.497131] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] gitlab # [ 6.400377] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.555647] erofs (device vdb): failed to read inode meta block (nid: 128711401): -4 gitlab # [ 6.422114] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 6.423290] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 6.425260] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.589639] erofs (device vdb): failed to read inode meta block (nid: 128819456): -4 gitlab # [ 6.457135] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 6.458626] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 6.464210] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.608644] ppdev: user-space parallel port driver gitlab # [ 6.629508] kvm_amd: TSC scaling supported gitlab # [ 6.630167] kvm_amd: Nested Virtualization enabled gitlab # [ 6.630828] kvm_amd: Nested Paging enabled gitlab # [ 6.631320] kvm_amd: LBR virtualization supported gitlab # [ 6.631925] kvm_amd: Virtual VMLOAD VMSAVE supported gitlab # [ 6.632568] kvm_amd: Virtual GIF supported gitlab # [ 6.633112] kvm_amd: Virtual NMI enabled gitlab # [ 6.528121] systemd-vconsole-setup[502]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 6.666812] EDAC MC: Ver: 3.0.0 gitlab # [ 6.531640] systemd[1]: Finished Virtual Console Setup. gitlab # [ 6.803362] systemd[1]: Mounting /run/wrappers... gitlab # [ 6.830054] systemd[1]: Mounted /run/wrappers. gitlab # [ 6.832069] systemd[1]: Reached target Local File Systems. gitlab # [ 6.833955] systemd[1]: Listening on Boot Loader Control Service Socket. gitlab # [ 6.836611] systemd[1]: Starting register-nix-paths.service... gitlab # [ 6.838905] systemd[1]: Starting Create SUID/SGID Wrappers... gitlab # [ 6.840940] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 6.843838] systemd[1]: Starting Save Transient machine-id to Disk... gitlab # [ 6.845966] systemd[1]: Starting Create System Files and Directories... gitlab # [ 6.879890] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. gitlab # [ 6.882947] systemd[1]: Finished Save Transient machine-id to Disk. gitlab # [ 6.890921] systemd[1]: Finished Create System Files and Directories. gitlab # [ 6.896105] systemd[1]: Starting Rebuild Journal Catalog... gitlab # [ 6.898480] systemd[1]: Starting Record System Boot/Shutdown in UTMP... gitlab # [ 6.918126] systemd[1]: Finished Record System Boot/Shutdown in UTMP. gitlab # [ 6.934224] systemd[1]: Finished Rebuild Journal Catalog. gitlab # [ 6.935667] systemd[1]: Starting Update is Completed... gitlab # [ 6.950059] systemd[1]: Finished Update is Completed. gitlab # [ 6.991243] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. gitlab # [ 6.992980] systemd[1]: Finished Create SUID/SGID Wrappers. gitlab # [ 7.228388] systemd[1]: Finished register-nix-paths.service. gitlab # [ 7.230607] systemd[1]: Reached target System Initialization. gitlab # [ 7.232713] systemd[1]: Started Discard unused filesystem blocks once a week. gitlab # [ 7.235318] systemd[1]: Started logrotate.timer. gitlab # [ 7.237168] systemd[1]: Started Daily Cleanup of Temporary Directories. gitlab # [ 7.239533] systemd[1]: Reached target Timer Units. gitlab # [ 7.241463] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 7.243725] systemd[1]: Listening on Nix Daemon Socket. gitlab # [ 7.245694] systemd[1]: Starting nixbot CI service socket... gitlab # [ 7.247748] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. gitlab # [ 7.250702] systemd[1]: Listening on nixbot CI service socket. gitlab # [ 7.252811] systemd[1]: Reached target Socket Units. gitlab # [ 7.254756] systemd[1]: Starting D-Bus System Message Bus... gitlab # [ 7.298914] dbus-broker-launch[584]: Looking up NSS user entry for 'systemd-timesync'... gitlab # [ 7.302189] dbus-broker-launch[584]: NSS returned no entry for 'systemd-timesync' gitlab # [ 7.304986] dbus-broker-launch[584]: Invalid user-name in /nix/store/37yal9w16hfdxjh64sww8gn81f4hma6i-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" gitlab # [ 7.310632] systemd[1]: Started D-Bus System Message Bus. gitlab # [ 7.324154] dbus-broker-launch[584]: Ready gitlab # [ 7.326904] systemd[1]: Reached target Basic System. gitlab # [ 7.329887] systemd[1]: Started backdoor.service. gitlab # [ 7.333176] systemd[1]: Starting gitlab-config.service... gitlab # [ 7.337167] systemd[1]: Starting Import lastlog data into lastlog2 database... gitlab # [ 7.353995] systemd[1]: Starting Logrotate configuration check... gitlab # [ 7.358642] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.362969] systemd[1]: Starting Post-Boot Actions... gitlab # [ 7.372665] systemd[1]: Started Reset console on configuration changes. gitlab # [ 7.379135] systemd[1]: Starting resolvconf update... gitlab # connecting to host... gitlab # [ 7.396516] logrotate[589]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. gitlab: Guest shell says: b'Spawning backdoor root shell...\n' gitlab # [ 7.399774] logrotate[589]: reading config file /nix/store/y3500wq742i80l631mh8nc8rchpsfb9q-logrotate.conf gitlab # [ 7.401244] logrotate[589]: note: 'monthly' overrides previously specified 'weekly' gitlab # [ 7.403179] systemd[1]: Finished Post-Boot Actions. gitlab: connected to guest root shell gitlab # [ 7.404147] nsncd[590]: Sep 04 05:31:55.549 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab: (connecting took 8.07 seconds) gitlab # [ 7.406397] logrotate[589]: Reading state from file: /var/lib/logrotate.status gitlab: (finished: waiting for the VM to finish booting, in 8.07 seconds) gitlab # [ 7.406979] logrotate[589]: state file /var/lib/logrotate.status does not exist gitlab # [ 7.407660] logrotate[589]: Allocating hash table for state file, size 64 entries gitlab # [ 7.408097] logrotate[589]: Handling 4 logs gitlab # [ 7.408511] logrotate[589]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.408962] logrotate[589]: considering log /var/log/btmp gitlab # [ 7.409883] logrotate[589]: Creating new state gitlab # [ 7.410446] logrotate[589]: Now: 2026-09-04 05:31 gitlab # [ 7.410918] logrotate[589]: Last rotated at 2026-09-04 05:00 gitlab # [ 7.411510] logrotate[589]: log does not need rotating (log has already been rotated) gitlab # [ 7.411981] logrotate[589]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.412601] logrotate[589]: considering log /var/log/wtmp gitlab # [ 7.413038] logrotate[589]: Creating new state gitlab # [ 7.413458] logrotate[589]: Now: 2026-09-04 05:31 gitlab # [ 7.413933] logrotate[589]: Last rotated at 2026-09-04 05:00 gitlab # [ 7.414514] logrotate[589]: log does not need rotating (log has already been rotated) gitlab # [ 7.414987] logrotate[589]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed gitlab # [ 7.415563] logrotate[589]: switching euid from 0 to 165 and egid from 0 to 165 (pid 589) gitlab # [ 7.416061] logrotate[589]: considering log /var/gitlab/state/log/*.log gitlab # [ 7.416478] logrotate[589]: log /var/gitlab/state/log/*.log does not exist -- skipping gitlab # [ 7.416807] logrotate[589]: Creating new state gitlab # [ 7.417719] logrotate[589]: switching euid from 165 to 0 and egid from 165 to 0 (pid 589) gitlab # [ 7.418397] logrotate[589]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed gitlab # [ 7.418912] logrotate[589]: switching euid from 0 to 60 and egid from 0 to 60 (pid 589) gitlab # [ 7.419556] logrotate[589]: considering log /var/log/nginx/*.log gitlab # [ 7.420094] logrotate[589]: log /var/log/nginx/*.log does not exist -- skipping gitlab # [ 7.420565] logrotate[589]: Creating new state gitlab # [ 7.421089] logrotate[589]: not running postrotate script, since no logs were rotated gitlab # [ 7.421562] logrotate[589]: switching euid from 60 to 0 and egid from 60 to 0 (pid 589) gitlab # [ 7.422789] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.423831] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.424793] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.425538] systemd[1]: Starting User Login Management... gitlab # [ 7.426690] systemd[1]: Finished Logrotate configuration check. gitlab # [ 7.427251] systemd[1]: Finished Import lastlog data into lastlog2 database. gitlab # [ 7.469160] systemd-logind[618]: New seat seat0. gitlab # [ 7.473673] systemd-logind[618]: Watching system buttons on /dev/input/event2 (Power Button) gitlab # [ 7.474606] systemd-logind[618]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) gitlab # [ 7.475176] systemd-logind[618]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) gitlab # [ 7.475670] systemd[1]: Started User Login Management. gitlab # [ 7.477202] systemd[1]: Starting linger-users.service... gitlab # [ 7.487719] systemd[1]: Stopped target Host and Network Name Lookups. gitlab # [ 7.488609] systemd[1]: Stopping Host and Network Name Lookups... gitlab # [ 7.489245] systemd[1]: Stopped target User and Group Name Lookups. gitlab # [ 7.489773] systemd[1]: Stopping User and Group Name Lookups... gitlab # [ 7.491191] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... gitlab # [ 7.494199] systemd[1]: nscd.service: Deactivated successfully. gitlab # [ 7.494861] systemd[1]: Stopped Name Service Cache Daemon (nsncd). gitlab # [ 7.495501] systemd[1]: nscd.service: Consumed 22ms CPU time over 136ms wall clock time, 7.3M memory peak, 5.5M read from disk. gitlab # [ 7.499203] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.502761] systemd[1]: linger-users.service: Deactivated successfully. gitlab # [ 7.505038] systemd[1]: Finished linger-users.service. gitlab # [ 7.514925] nsncd[681]: Sep 04 05:31:55.664 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.517170] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.518283] systemd[1]: Finished resolvconf update. gitlab # [ 7.518911] systemd[1]: Reached target Preparation for Network. gitlab # [ 7.519584] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.519756] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.521659] systemd[1]: Starting DHCP Client... gitlab # [ 7.522849] systemd[1]: Starting Address configuration of eth1... gitlab # [ 7.524828] systemd[1]: Starting Extra networking commands.... gitlab # [ 7.547363] network-addresses-eth1-start[707]: adding address 192.168.1.1/24... done gitlab # [ 7.550347] network-addresses-eth1-start[707]: adding address 2001:db8:1::1/64... done gitlab # [ 7.555550] systemd[1]: Finished Address configuration of eth1. gitlab # [ 7.584244] systemd[1]: Finished Extra networking commands.. gitlab # [ 7.585341] dhcpcd[716]: dhcpcd-10.3.2 starting gitlab # [ 7.586468] systemd[1]: Reached target Network. gitlab # [ 7.588417] dhcpcd[772]: dev: loaded udev gitlab # [ 7.589630] systemd[1]: Starting gitlab-workhorse.service... gitlab # [ 7.591917] systemd[1]: Starting Nginx Web Server... gitlab # [ 7.595086] systemd[1]: Starting PostgreSQL Server... gitlab # [ 7.733232] 8021q: 802.1Q VLAN Support v1.8 gitlab # [ 7.596684] systemd[1]: Starting Redis Server - redis-gitlab... gitlab # [ 7.735722] 8021q: adding VLAN 0 to HW filter on device eth1 gitlab # [ 7.600539] systemd[1]: Starting Permit User Sessions... gitlab # [ 7.616798] systemd[1]: Finished Permit User Sessions. gitlab # [ 7.621752] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. gitlab # [ 7.631863] systemd[1]: Started Getty on tty1. gitlab # [ 7.632926] systemd[1]: Reached target Login Prompts. gitlab # [ 7.829817] cfg80211: Loading compiled-in X.509 certificates for regulatory database gitlab # [ 7.834981] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' gitlab # [ 7.836931] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' gitlab # [ 7.839077] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 gitlab # [ 7.840453] cfg80211: failed to load regulatory.db gitlab # [ 7.866341] 8021q: adding VLAN 0 to HW filter on device eth0 gitlab # [ 7.731806] dhcpcd[772]: eth0: waiting for carrier gitlab # [ 7.732653] dhcpcd[772]: eth0: carrier acquired gitlab # [ 7.736288] dhcpcd[772]: DUID 00:01:00:01:32:2d:14:4b:52:54:00:12:34:56 gitlab # [ 7.737057] dhcpcd[772]: eth0: IAID 00:12:34:56 gitlab # [ 7.737517] dhcpcd[772]: eth0: adding address fe80::5054:ff:fe12:3456 gitlab # [ 7.739719] postgresql-pre-start[793]: The files belonging to this database system will be owned by user "postgres". gitlab # [ 7.740635] postgresql-pre-start[793]: This user must also own the server process. gitlab # [ 7.744883] postgresql-pre-start[793]: The database cluster will be initialized with locale "en_US.UTF-8". gitlab # [ 7.745555] postgresql-pre-start[793]: The default database encoding has accordingly been set to "UTF8". gitlab # [ 7.746108] postgresql-pre-start[793]: The default text search configuration will be set to "english". gitlab # [ 7.746572] postgresql-pre-start[793]: Data page checksums are enabled. gitlab # [ 7.747337] postgresql-pre-start[793]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok gitlab # [ 7.748127] postgresql-pre-start[793]: creating subdirectories ... ok gitlab # [ 7.748789] postgresql-pre-start[793]: selecting dynamic shared memory implementation ... posix gitlab # [ 7.751947] redis-server[799]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. gitlab # [ 7.754402] redis-server[799]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo gitlab # [ 7.755781] redis-server[799]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=799, just started gitlab # [ 7.756876] systemd[1]: Started Redis Server - redis-gitlab. gitlab # [ 7.757629] redis-server[799]: Configuration loaded gitlab # [ 7.758340] redis[799]: monotonic clock: POSIX clock_gettime gitlab # [ 7.759189] redis[799]: Running mode=standalone, port=0. gitlab # [ 7.760212] redis[799]: Server initialized gitlab # [ 7.761568] redis[799]: Ready to accept connections unix gitlab # [ 7.762270] redis[799]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. gitlab # [ 7.774362] nginx-pre-start[802]: nginx: the configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf syntax is ok gitlab # [ 7.776356] nginx-pre-start[802]: nginx: configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf test is successful gitlab # [ 7.780116] systemd[1]: Started Nginx Web Server. gitlab # [ 7.790283] postgresql-pre-start[793]: selecting default "max_connections" ... 100 gitlab # [ 7.836569] postgresql-pre-start[793]: selecting default "shared_buffers" ... 128MB gitlab # [ 8.085066] systemd[1]: Started gitlab-workhorse.service. gitlab # [ 8.280411] postgresql-pre-start[793]: selecting default time zone ... UTC gitlab # [ 8.281949] postgresql-pre-start[793]: creating configuration files ... ok gitlab # [ 8.292568] gitlab-workhorse[832]: time="2026-09-04T05:31:56Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4 gitlab # [ 8.293665] gitlab-workhorse[832]: time="2026-09-04T05:31:56Z" level=info msg="Using redis/go-redis" gitlab # [ 8.294680] gitlab-workhorse[832]: time="2026-09-04T05:31:56Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix gitlab # [ 8.434992] postgresql-pre-start[793]: running bootstrap script ... ok gitlab # [ 8.856975] dhcpcd[772]: eth0: soliciting a DHCP lease gitlab # [ 8.857991] postgresql-pre-start[793]: performing post-bootstrap initialization ... ok gitlab # [ 9.000069] NET: Registered PF_PACKET protocol family gitlab # [ 8.878947] dhcpcd[772]: eth0: offered 10.0.2.15 from 10.0.2.2 gitlab # [ 8.900295] dhcpcd[772]: eth0: probing address 10.0.2.15/24 gitlab # [ 9.112934] postgresql-pre-start[793]: syncing data to disk ... ok gitlab # [ 9.115380] postgresql-pre-start[793]: initdb: warning: enabling "trust" authentication for local connections gitlab # [ 9.118659] postgresql-pre-start[793]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. gitlab # [ 9.123737] postgresql-pre-start[793]: Success. You can now start the database server using: gitlab # [ 9.126562] postgresql-pre-start[793]: pg_ctl -D /var/lib/postgresql/18 -l logfile start gitlab # [ 9.199104] postgres[858]: [858] LOG: starting PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit gitlab # [ 9.203157] postgres[858]: [858] LOG: listening on IPv6 address "::1", port 5432 gitlab # [ 9.205864] postgres[858]: [858] LOG: listening on IPv4 address "127.0.0.1", port 5432 gitlab # [ 9.208535] postgres[858]: [858] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" gitlab # [ 9.220222] postgres[867]: [867] LOG: database system was shut down at 2026-09-04 05:31:56 GMT gitlab # [ 9.226520] postgres[858]: [858] LOG: database system is ready to accept connections gitlab # [ 9.233287] systemd[1]: Started PostgreSQL Server. gitlab # [ 9.235697] systemd[1]: Starting PostgreSQL Setup Scripts... gitlab # [ 9.451375] postgresql-setup-start[878]: CREATE DATABASE gitlab # [ 9.479331] postgresql-setup-start[888]: CREATE ROLE gitlab # [ 9.490349] postgresql-setup-start[890]: ALTER DATABASE gitlab # [ 9.514120] postgresql-setup-start[895]: CREATE ROLE gitlab # [ 9.517729] systemd[1]: Finished PostgreSQL Setup Scripts. gitlab # [ 9.518883] systemd[1]: Reached target PostgreSQL. gitlab # [ 9.523068] systemd[1]: Starting gitlab-postgresql.service... gitlab # [ 9.525870] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[883]: mkdir -p /var/gitlab/state/home/.ssh: OK gitlab # [ 9.528721] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[883]: chmod 700 /var/gitlab/state/home/.ssh: OK gitlab # [ 9.570085] systemd[1]: Finished gitlab-config.service. gitlab # [ 9.571833] systemd[1]: Started gitaly.service. gitlab # [ 9.573146] systemd[1]: Starting gitlab-db-config.service... gitlab # [ 9.623656] gitlab-postgresql-start[923]: CREATE DATABASE gitlab # [ 9.666169] gitlab-postgresql-start[933]: CREATE EXTENSION gitlab # [ 9.748608] gitlab-postgresql-start[935]: CREATE EXTENSION gitlab # [ 9.752661] systemd[1]: Finished gitlab-postgresql.service. gitlab # [ 9.786728] dhcpcd[772]: eth0: soliciting an IPv6 router gitlab # [ 9.788296] dhcpcd[772]: eth0: Router Advertisement from fe80::2 gitlab # [ 9.788917] dhcpcd[772]: eth0: adding address fec0::5054:ff:fe12:3456/64 gitlab # [ 9.789675] dhcpcd[772]: eth0: adding route to fec0::/64 gitlab # [ 9.790437] dhcpcd[772]: eth0: adding default route via fe80::2 gitlab # [ 10.073632] gitaly[926]: time="2026-09-04T05:31:58.222Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/ymnlr20kjfmhkzg7i9hp25h3ii8qnn0d-gitaly.toml' and fix the errors that are printed." pid=926 gitlab # [ 10.075310] gitaly[926]: time="2026-09-04T05:31:58.222Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=926 gitlab # [ 10.076359] gitaly[926]: time="2026-09-04T05:31:58.222Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=926 gitlab # [ 10.077545] gitaly[926]: time="2026-09-04T05:31:58.222Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=926 gitlab # [ 10.078964] gitaly[926]: time="2026-09-04T05:31:58.222Z" level=info msg="Starting Gitaly" pid=926 version= gitlab # [ 10.237399] gitaly[926]: time="2026-09-04T05:31:58.386Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=163 pid=926 gitlab # [ 10.239308] gitaly[926]: time="2026-09-04T05:31:58.386Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=926 gitlab # [ 10.241050] gitaly[926]: time="2026-09-04T05:31:58.386Z" level=info msg="finished initializing command factory" duration_ms=0 pid=926 gitlab # [ 10.242807] gitaly[926]: time="2026-09-04T05:31:58.386Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-926/git-exec-1792189452.d/git pid=926 gitlab # [ 10.244896] gitaly[926]: time="2026-09-04T05:31:58.393Z" level=info msg="finished detecting git version" duration_ms=6 pid=926 gitlab # [ 10.246641] gitaly[926]: time="2026-09-04T05:31:58.393Z" level=info msg="using Git version" pid=926 version=2.53.g gitlab # [ 10.248264] gitaly[926]: time="2026-09-04T05:31:58.393Z" level=info msg="clearing disk cache object folder" pid=926 storage=default gitlab # [ 10.250035] gitaly[926]: time="2026-09-04T05:31:58.393Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3769985306 pid=926 storage=default gitlab # [ 10.252656] gitaly[926]: time="2026-09-04T05:31:58.393Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3769985306 pid=926 storage=default gitlab # [ 10.255613] gitaly[926]: time="2026-09-04T05:31:58.393Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=926 gitlab # [ 10.258398] gitaly[926]: time="2026-09-04T05:31:58.393Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=926 gitlab # [ 10.261177] gitaly[926]: time="2026-09-04T05:31:58.393Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3769985306 clear_duration_ms=0 pid=926 storage=default gitlab # [ 10.277556] gitaly[926]: time="2026-09-04T05:31:58.426Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=926 gitlab # [ 10.280123] gitaly[926]: time="2026-09-04T05:31:58.426Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=926 schema=unix gitlab # [ 10.280823] gitaly[926]: time="2026-09-04T05:31:58.427Z" level=info msg="listening at address" address=/run/gitaly/gitaly-926/sock.d/intern pid=926 schema=unix gitlab # [ 10.281538] gitaly[926]: time="2026-09-04T05:31:58.427Z" level=info msg="finished tempdir cleaner walk" pid=926 storage=default time_ms=0 gitlab # [ 10.282132] gitaly[926]: time="2026-09-04T05:31:58.426Z" level=info msg="counting repositories" pid=926 gitlab # [ 10.282496] gitaly[926]: time="2026-09-04T05:31:58.427Z" level=info msg="starting to count repositories" pid=926 storage_path=/var/gitlab/state/repositories gitlab # [ 10.283094] gitaly[926]: time="2026-09-04T05:31:58.427Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=926 storage_path=/var/gitlab/state/repositories gitlab # [ 10.283609] gitaly[926]: time="2026-09-04T05:31:58.427Z" level=info msg="completed counting all repositories" counting_duration=0 pid=926 gitlab # [ 10.284156] gitaly[926]: time="2026-09-04T05:31:58.427Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=926 gitlab # [ 10.284767] gitaly[926]: time="2026-09-04T05:31:58.428Z" level=info msg="Started Gitaly" duration_ms=205 pid=926 gitlab # [ 10.285582] gitaly[926]: time="2026-09-04T05:31:58.428Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=926 gitlab # [ 10.286154] gitaly[926]: time="2026-09-04T05:31:58.428Z" level=info msg="maintenance: daily scheduled" pid=926 scheduled="2026-09-04 12:00:00 +0000 UTC" gitlab # [ 11.138647] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[929]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 11.144528] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[929]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 11.190227] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[929]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 12.124285] gitaly[926]: time="2026-09-04T05:32:00.272Z" level=info msg="License database preloaded" duration_ms=2050 pid=926 gitlab # [ 13.496709] dhcpcd[772]: eth0: leased 10.0.2.15 for 86400 seconds gitlab # [ 13.497962] dhcpcd[772]: eth0: adding route to 10.0.2.0/24 gitlab # [ 13.499720] dhcpcd[772]: eth0: adding default route via 10.0.2.2 gitlab # [ 13.624172] systemd[1]: Started DHCP Client. gitlab # [ 13.626371] systemd[1]: Reached target Network is Online. gitlab # [ 40.722245] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[929]: W, [2026-09-04T05:32:28.870846 #929] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 42.719405] postgres[1315]: [1315] ERROR: relation "feature_gates" does not exist at character 523 gitlab # [ 42.722523] postgres[1315]: [1315] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), gitlab # [ 42.725578] postgres[1315]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, gitlab # [ 42.728781] postgres[1315]: c.collname, col_description(a.attrelid, a.attnum) AS comment, gitlab # [ 42.731950] postgres[1315]: attidentity AS identity, gitlab # [ 42.733943] postgres[1315]: attgenerated as attgenerated gitlab # [ 42.736077] postgres[1315]: FROM pg_attribute a gitlab # [ 42.737771] postgres[1315]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum gitlab # [ 42.740694] postgres[1315]: LEFT JOIN pg_type t ON a.atttypid = t.oid gitlab # [ 42.742914] postgres[1315]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation gitlab # [ 42.746322] postgres[1315]: WHERE a.attrelid = '"feature_gates"'::regclass gitlab # [ 42.748639] postgres[1315]: AND a.attnum > 0 AND NOT a.attisdropped gitlab # [ 42.750831] postgres[1315]: ORDER BY a.attnum gitlab # [ 42.752510] postgres[1315]: gitlab # [ 45.999190] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1348]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping gitlab # [ 46.003873] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1348]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping gitlab # [ 48.353871] gitlab-workhorse[832]: time="2026-09-04T05:32:36Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 58.363322] gitlab-workhorse[832]: time="2026-09-04T05:32:46Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 60.504613] postgres[1423]: [1423] ERROR: relation "table_without_models" does not exist gitlab # [ 60.506110] postgres[1423]: [1423] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 68.372744] gitlab-workhorse[832]: time="2026-09-04T05:32:56Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 72.255910] systemd[1]: Finished gitlab-db-config.service. gitlab # [ 72.259749] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 72.262971] systemd[1]: Starting gitlab.service... gitlab # [ 72.620330] bundle[1590]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 72.621772] bundle[1590]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 72.622582] bundle[1590]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 75.290197] sidekiq-cluster[1589]: {"severity":"INFO","time":"2026-09-04T05:33:03.422Z","message":"Starting cluster with 1 processes"} gitlab # [ 75.382393] bundle[1590]: {"timestamp":"2026-09-04T05:33:03.512Z","pid":1590,"message":"* Enabling systemd notification integration"} gitlab # [ 75.383150] bundle[1590]: {"timestamp":"2026-09-04T05:33:03.531Z","pid":1590,"message":"Puma starting in cluster mode..."} gitlab # [ 75.383648] bundle[1590]: {"timestamp":"2026-09-04T05:33:03.531Z","pid":1590,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"} gitlab # [ 75.384973] bundle[1590]: {"timestamp":"2026-09-04T05:33:03.531Z","pid":1590,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 75.385817] bundle[1590]: {"timestamp":"2026-09-04T05:33:03.531Z","pid":1590,"message":"* Min threads: 0"} gitlab # [ 75.386706] bundle[1590]: {"timestamp":"2026-09-04T05:33:03.531Z","pid":1590,"message":"* Max threads: 4"} gitlab # [ 75.387615] bundle[1590]: {"timestamp":"2026-09-04T05:33:03.531Z","pid":1590,"message":"* Environment: production"} gitlab # [ 75.388872] bundle[1590]: {"timestamp":"2026-09-04T05:33:03.531Z","pid":1590,"message":"* Master PID: 1590"} gitlab # [ 75.389780] bundle[1590]: {"timestamp":"2026-09-04T05:33:03.531Z","pid":1590,"message":"* Workers: 2"} gitlab # [ 75.390766] bundle[1590]: {"timestamp":"2026-09-04T05:33:03.531Z","pid":1590,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} gitlab # [ 75.391953] bundle[1590]: {"timestamp":"2026-09-04T05:33:03.531Z","pid":1590,"message":"* Preloading application"} gitlab # [ 78.381933] gitlab-workhorse[832]: time="2026-09-04T05:33:06Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 88.383171] gitlab-workhorse[832]: time="2026-09-04T05:33:16Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 97.477409] bundle[1590]: W, [2026-09-04T05:33:25.626299 #1590] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 98.384364] gitlab-workhorse[832]: time="2026-09-04T05:33:26Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 99.987945] sidekiq-cluster[1611]: 2026-09-04T05:33:28.136Z pid=1611 tid=wkj INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1611, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>440377344} gitlab # [ 100.152668] sidekiq-cluster[1611]: W, [2026-09-04T05:33:28.301776 #1611] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 100.191814] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:28.341Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 102.042688] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.191Z","message":"GitLab reliable fetch activated!"} gitlab # [ 102.046396] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.191Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 102.063722] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.212Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} gitlab # [ 102.065346] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.214Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"} gitlab # [ 102.067350] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.216Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} gitlab # [ 102.069362] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.218Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} gitlab # [ 102.071432] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.220Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} gitlab # [ 102.073524] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.222Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} gitlab # [ 102.075305] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.224Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} gitlab # [ 102.076862] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.226Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} gitlab # [ 102.078427] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.227Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} gitlab # [ 102.079983] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.229Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} gitlab # [ 102.081565] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.230Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} gitlab # [ 102.083595] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.232Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} gitlab # [ 102.085285] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.234Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} gitlab # [ 102.086684] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.235Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} gitlab # [ 102.088366] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.237Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} gitlab # [ 102.089984] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.239Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} gitlab # [ 102.092732] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.241Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} gitlab # [ 102.094847] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.244Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} gitlab # [ 102.097234] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.246Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} gitlab # [ 102.099304] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.248Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} gitlab # [ 102.101428] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.250Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} gitlab # [ 102.103352] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.252Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} gitlab # [ 102.105597] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.254Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} gitlab # [ 102.107114] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.256Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} gitlab # [ 102.108977] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.258Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} gitlab # [ 102.111201] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.260Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} gitlab # [ 102.113257] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.262Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} gitlab # [ 102.115416] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.264Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} gitlab # [ 102.117631] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.266Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} gitlab # [ 102.119869] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.269Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} gitlab # [ 102.121401] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.270Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} gitlab # [ 102.123572] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.272Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} gitlab # [ 102.125630] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.274Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} gitlab # [ 102.127825] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.277Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} gitlab # [ 102.129592] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.278Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} gitlab # [ 102.131333] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.280Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} gitlab # [ 102.133101] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.282Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} gitlab # [ 102.135438] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.284Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} gitlab # [ 102.136929] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.286Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} gitlab # [ 102.138768] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.288Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} gitlab # [ 102.140512] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.289Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} gitlab # [ 102.142119] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.291Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} gitlab # [ 102.143768] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.293Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} gitlab # [ 102.145615] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.294Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} gitlab # [ 102.147680] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.296Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} gitlab # [ 102.149717] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.298Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} gitlab # [ 102.151364] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.300Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} gitlab # [ 102.153366] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.302Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} gitlab # [ 102.155717] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.304Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} gitlab # [ 102.157458] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.306Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} gitlab # [ 102.159518] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.308Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} gitlab # [ 102.161608] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.310Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} gitlab # [ 102.169877] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.319Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} gitlab # [ 102.172337] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.321Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} gitlab # [ 102.174175] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.323Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} gitlab # [ 102.176189] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.325Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} gitlab # [ 102.178121] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.327Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} gitlab # [ 102.180321] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.329Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} gitlab # [ 102.181780] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.331Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} gitlab # [ 102.183834] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.333Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} gitlab # [ 102.185753] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.335Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} gitlab # [ 102.187291] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.336Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} gitlab # [ 102.189570] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.338Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} gitlab # [ 102.191703] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.340Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} gitlab # [ 102.193686] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.342Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} gitlab # [ 102.195780] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.345Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} gitlab # [ 102.197397] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.346Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} gitlab # [ 102.199190] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.348Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} gitlab # [ 102.200686] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.349Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} gitlab # [ 102.202461] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.351Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} gitlab # [ 102.203853] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.353Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} gitlab # [ 102.206620] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.355Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} gitlab # [ 102.210466] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.359Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} gitlab # [ 102.213623] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.362Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} gitlab # [ 102.216933] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.366Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} gitlab # [ 102.220267] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.369Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} gitlab # [ 102.224276] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.373Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} gitlab # [ 102.227791] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.377Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} gitlab # [ 102.231263] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.380Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} gitlab # [ 102.234220] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.383Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} gitlab # [ 102.237175] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.386Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} gitlab # [ 102.240108] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.389Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} gitlab # [ 102.243977] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.393Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} gitlab # [ 102.247022] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.396Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} gitlab # [ 102.251194] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.400Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} gitlab # [ 102.254305] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.403Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} gitlab # [ 102.258229] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.407Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} gitlab # [ 102.261919] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.411Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} gitlab # [ 102.264224] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.413Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} gitlab # [ 102.266071] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.415Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} gitlab # [ 102.268285] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.417Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} gitlab # [ 102.270391] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.419Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} gitlab # [ 102.272508] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.421Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} gitlab # [ 102.274589] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.423Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} gitlab # [ 102.277072] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.426Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} gitlab # [ 102.279718] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.429Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} gitlab # [ 102.282038] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.431Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} gitlab # [ 102.284125] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.433Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} gitlab # [ 102.294232] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.443Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} gitlab # [ 102.296512] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.445Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} gitlab # [ 102.298693] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.448Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} gitlab # [ 102.301437] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.450Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} gitlab # [ 102.304308] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.453Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} gitlab # [ 102.307071] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.456Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} gitlab # [ 102.309988] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.459Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} gitlab # [ 102.312100] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.461Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} gitlab # [ 102.314385] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.463Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} gitlab # [ 102.317445] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.466Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} gitlab # [ 102.319737] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:30.469Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} gitlab # [ 104.677782] sidekiq-cluster[1611]: InitializerConnections Query: SET client_min_messages TO 'warning' gitlab # [ 104.680504] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 104.684341] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 104.689256] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 104.691109] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 104.691787] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start' gitlab # [ 104.696518] sidekiq-cluster[1611]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 104.697331] sidekiq-cluster[1611]: InitializerConnections Query: SET standard_conforming_strings = on gitlab # [ 104.698399] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 104.701215] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 104.701691] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 104.702846] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 104.703967] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start' gitlab # [ 104.705093] sidekiq-cluster[1611]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 104.705846] sidekiq-cluster[1611]: InitializerConnections Query: SET intervalstyle = iso_8601 gitlab # [ 104.706990] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 104.707702] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 104.708838] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 104.709954] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 104.711192] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start' gitlab # [ 104.712075] sidekiq-cluster[1611]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 104.761310] sidekiq-cluster[1611]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT t.oid, t.typname gitlab # [ 104.765557] sidekiq-cluster[1611]: FROM pg_type as t gitlab # [ 104.767128] sidekiq-cluster[1611]: WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'numeric', 'bool', 'timestamp', 'timestamptz') gitlab # [ 104.768585] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 104.773996] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 104.775538] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 104.776943] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 104.778451] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start' gitlab # [ 104.779940] sidekiq-cluster[1611]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 104.781747] sidekiq-cluster[1611]: InitializerConnections Query: SET SESSION timezone TO 'UTC' gitlab # [ 104.782964] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 104.784179] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 104.785096] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 104.785977] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 104.787203] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start' gitlab # [ 104.788122] sidekiq-cluster[1611]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 104.789060] sidekiq-cluster[1611]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT "feature_gates"."key", "feature_gates"."value" FROM "feature_gates" WHERE "feature_gates"."feature_key" = 'per_thread_db_connection_pool_metrics' gitlab # [ 104.789958] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 104.790812] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 104.791982] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 104.793175] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 104.794066] sidekiq-cluster[1611]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start' gitlab # [ 104.794944] sidekiq-cluster[1611]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 108.385508] gitlab-workhorse[832]: time="2026-09-04T05:33:36Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 118.394768] gitlab-workhorse[832]: time="2026-09-04T05:33:46Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 126.313420] postgres[2049]: [2049] ERROR: relation "table_without_models" does not exist gitlab # [ 126.315449] postgres[2049]: [2049] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 127.724081] postgres[2060]: [2060] ERROR: relation "table_without_models" does not exist gitlab # [ 127.725376] postgres[2060]: [2060] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 128.397373] gitlab-workhorse[832]: time="2026-09-04T05:33:56Z" level=error correlation_id= duration_ms=1 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 128.922789] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:57.069Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 128.925256] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:57.074Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 128.927245] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:57.074Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 128.929070] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:57.074Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 128.934701] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:57.081Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 130.261497] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:58.410Z","message":"Cleaning working queues"} gitlab # [ 130.263875] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:58.412Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"d19b92166d18eedc5e114c80","created_at":"2026-09-04T05:33:00.014Z","trace_propagation_headers":{"sentry-trace":"da8417bb5b1941fc994c1a2121312415-f0093dcb30a346fd","baggage":"sentry-trace_id=da8417bb5b1941fc994c1a2121312415,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"059d8675f6988fde0c7ba35c78775446","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-04T05:33:00.021Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-d19b92166d18eedc5e114c80: start","job_status":"start","queue_duration_s":58.391488,"scheduling_latency_s":58.39157} gitlab # [ 130.442651] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:33:58.591Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"d19b92166d18eedc5e114c80","created_at":"2026-09-04T05:33:00.014Z","trace_propagation_headers":{"sentry-trace":"da8417bb5b1941fc994c1a2121312415-f0093dcb30a346fd","baggage":"sentry-trace_id=da8417bb5b1941fc994c1a2121312415,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"059d8675f6988fde0c7ba35c78775446","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-04T05:33:00.021Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-d19b92166d18eedc5e114c80: done: 0.178754 sec","job_status":"done","queue_duration_s":58.391488,"scheduling_latency_s":58.39157,"redis_calls":29,"redis_duration_s":0.006073,"redis_read_bytes":282,"redis_write_bytes":3632,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.003058,"redis_feature_flag_read_bytes":272,"redis_feature_flag_write_bytes":2385,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002015,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":15,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.074358,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.096114731,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.178754,"completed_at":"2026-09-04T05:33:58.591Z","load_balancing_strategy":"primary","db_duration_s":0.065197,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 130.535689] bundle[1590]: {"timestamp":"2026-09-04T05:33:58.684Z","pid":1590,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} gitlab # [ 130.537100] bundle[1590]: {"timestamp":"2026-09-04T05:33:58.684Z","pid":1590,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} gitlab # [ 130.538252] bundle[1590]: {"timestamp":"2026-09-04T05:33:58.684Z","pid":1590,"message":"! #\u003cThread:0x0000742da56909a0 /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} gitlab # [ 130.539459] bundle[1590]: {"timestamp":"2026-09-04T05:33:58.684Z","pid":1590,"message":"! #\u003cThread:0x0000742da58db368 /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} gitlab # [ 130.540994] bundle[1590]: {"timestamp":"2026-09-04T05:33:58.684Z","pid":1590,"message":"Use Ctrl-C to stop"} gitlab # [ 130.542176] bundle[1590]: {"timestamp":"2026-09-04T05:33:58.684Z","pid":1590,"message":"Sending status to systemd every 1.0 sec"} gitlab # [ 130.899187] bundle[1590]: {"timestamp":"2026-09-04T05:33:59.048Z","pid":1590,"message":"- Worker 0 (PID: 2161) booted in 0.3s, phase: 0"} gitlab # [ 130.918488] bundle[1590]: {"timestamp":"2026-09-04T05:33:59.067Z","pid":1590,"message":"- Worker 1 (PID: 2165) booted in 0.3s, phase: 0"} gitlab # [ 130.923075] systemd[1]: Started gitlab.service. gitlab # [ 130.924832] systemd[1]: Reached target Common target for all GitLab services.. gitlab # [ 130.927334] systemd[1]: Starting setup-gitlab.service... gitlab # [ 130.945296] setup-gitlab-start[2189]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' gitlab # [ 133.198342] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:34:01 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 2229 gitlab # [ 133.203827] setup-gitlab-start[2189]: + TOKEN=nixbot-test-token-of-20-chars gitlab # [ 133.204628] setup-gitlab-start[2189]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n' gitlab # [ 133.224727] runuser[2211]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) gitlab # [ 133.613763] setup-gitlab-start[2212]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 133.615089] setup-gitlab-start[2212]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 133.616491] setup-gitlab-start[2212]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 134.184134] setup-gitlab-start[2212]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 134.186181] setup-gitlab-start[2212]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 134.187163] setup-gitlab-start[2212]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 143.845817] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:11.994Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"4d98c8fddfba1ae615653709","created_at":"2026-09-04T05:34:11.962Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb25ca37473c38529c3ab3a1ee8f02c8","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:11.992Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-4d98c8fddfba1ae615653709: start","job_status":"start","queue_duration_s":0.002095,"scheduling_latency_s":0.002135} gitlab # [ 143.877147] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.026Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"4d98c8fddfba1ae615653709","created_at":"2026-09-04T05:34:11.962Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb25ca37473c38529c3ab3a1ee8f02c8","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:11.992Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-4d98c8fddfba1ae615653709: done: 0.031765 sec","job_status":"done","queue_duration_s":0.002095,"scheduling_latency_s":0.002135,"redis_calls":13,"redis_duration_s":0.0025789999999999997,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000389,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001393,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000797,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020914,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.008895265000000001,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.031765,"completed_at":"2026-09-04T05:34:12.026Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002256,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 143.949820] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.098Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"4d59d01b3a7612dec414bd67","created_at":"2026-09-04T05:34:12.093Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"60baa7d247e8281e91c6932ae212c1fe","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.097Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-4d59d01b3a7612dec414bd67: start","job_status":"start","queue_duration_s":0.001029,"scheduling_latency_s":0.001059} gitlab # [ 144.044197] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.192Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"4d59d01b3a7612dec414bd67","created_at":"2026-09-04T05:34:12.093Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"60baa7d247e8281e91c6932ae212c1fe","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.097Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-4d59d01b3a7612dec414bd67: done: 0.094111 sec","job_status":"done","queue_duration_s":0.001029,"scheduling_latency_s":0.001059,"redis_calls":12,"redis_duration_s":0.027803,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.001774,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000376,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.023068,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002585,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015851,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.010542402,"duration_s":0.094111,"completed_at":"2026-09-04T05:34:12.192Z","load_balancing_strategy":"primary","db_duration_s":0.020026,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.046238] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.195Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"debe6782bf0239683f83c6b8","created_at":"2026-09-04T05:34:12.125Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0221ad9a81f923fc9f180c3f611b74d6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-04T05:34:12.135Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-debe6782bf0239683f83c6b8: start","job_status":"start","queue_duration_s":0.060119,"scheduling_latency_s":0.06015} gitlab # [ 144.109717] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.210Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 144.110419] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.220Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"debe6782bf0239683f83c6b8","created_at":"2026-09-04T05:34:12.125Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0221ad9a81f923fc9f180c3f611b74d6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-04T05:34:12.135Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-debe6782bf0239683f83c6b8: done: 0.024869 sec","job_status":"done","queue_duration_s":0.060119,"scheduling_latency_s":0.06015,"redis_calls":10,"redis_duration_s":0.002074,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000427,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001245,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000402,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015158,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000415136,"duration_s":0.024869,"completed_at":"2026-09-04T05:34:12.220Z","load_balancing_strategy":"primary","db_duration_s":0.000505,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.119203] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.221Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"9e70e1dae94c47d4c10fdd09","created_at":"2026-09-04T05:34:12.140Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1e89d37301675672bf7a670849c666b","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.144Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-9e70e1dae94c47d4c10fdd09: start","job_status":"start","queue_duration_s":0.077026,"scheduling_latency_s":0.077051} gitlab # [ 144.131723] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.244Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"9e70e1dae94c47d4c10fdd09","created_at":"2026-09-04T05:34:12.140Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1e89d37301675672bf7a670849c666b","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.144Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-9e70e1dae94c47d4c10fdd09: done: 0.022796 sec","job_status":"done","queue_duration_s":0.077026,"scheduling_latency_s":0.077051,"redis_calls":10,"redis_duration_s":0.0018800000000000002,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000351,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001144,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000385,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014475,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000319035,"duration_s":0.022796,"completed_at":"2026-09-04T05:34:12.244Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001882,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.175537] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.245Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"33cdf14bb5ec56c93aa80379","created_at":"2026-09-04T05:34:12.159Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"22e657e9eb71d89cd4ab144c7d4b5f3f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.163Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-33cdf14bb5ec56c93aa80379: start","job_status":"start","queue_duration_s":0.08141,"scheduling_latency_s":0.081423} gitlab # [ 144.178555] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.266Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"33cdf14bb5ec56c93aa80379","created_at":"2026-09-04T05:34:12.159Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"22e657e9eb71d89cd4ab144c7d4b5f3f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.163Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-33cdf14bb5ec56c93aa80379: done: 0.020782 sec","job_status":"done","queue_duration_s":0.08141,"scheduling_latency_s":0.081423,"redis_calls":19,"redis_duration_s":0.002513,"redis_read_bytes":21,"redis_write_bytes":4785,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000507,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000234,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.001323,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000449,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014441,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000260089,"duration_s":0.020782,"completed_at":"2026-09-04T05:34:12.266Z","load_balancing_strategy":"primary","db_duration_s":0.001165,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.216223] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.267Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"2b8df814926ded27e113f146","created_at":"2026-09-04T05:34:12.167Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"69425a23beaf824be30b267b4ea1b1ec","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.171Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-2b8df814926ded27e113f146: start","job_status":"start","queue_duration_s":0.095404,"scheduling_latency_s":0.095418} gitlab # [ 144.262331] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.280Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"2b8df814926ded27e113f146","created_at":"2026-09-04T05:34:12.167Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"69425a23beaf824be30b267b4ea1b1ec","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.171Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-2b8df814926ded27e113f146: done: 0.013445 sec","job_status":"done","queue_duration_s":0.095404,"scheduling_latency_s":0.095418,"redis_calls":12,"redis_duration_s":0.001608,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00054,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00076,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000308,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00757,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000225727,"duration_s":0.013445,"completed_at":"2026-09-04T05:34:12.280Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001166,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.263497] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.281Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ceaa2e5a315d490978fc027f","created_at":"2026-09-04T05:34:12.254Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-69ff20d0bbad4fec","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"22e657e9eb71d89cd4ab144c7d4b5f3f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.261Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ceaa2e5a315d490978fc027f: start","job_status":"start","queue_duration_s":0.019588,"scheduling_latency_s":0.019602} gitlab # [ 144.300868] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.296Z","message":"No existing merge request to be cleaned up."} gitlab # [ 144.301563] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.303Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ceaa2e5a315d490978fc027f","created_at":"2026-09-04T05:34:12.254Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-69ff20d0bbad4fec","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"22e657e9eb71d89cd4ab144c7d4b5f3f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.261Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ceaa2e5a315d490978fc027f: done: 0.022247 sec","job_status":"done","queue_duration_s":0.019588,"scheduling_latency_s":0.019602,"redis_calls":13,"redis_duration_s":0.001876,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00027,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000665,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000941,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015542,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000197511,"duration_s":0.022247,"completed_at":"2026-09-04T05:34:12.303Z","load_balancing_strategy":"primary","db_duration_s":0.002486,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.307639] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.304Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f5714fab493c0d22df9a05ee","created_at":"2026-09-04T05:34:12.254Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-69ff20d0bbad4fec","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"22e657e9eb71d89cd4ab144c7d4b5f3f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.261Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f5714fab493c0d22df9a05ee: start","job_status":"start","queue_duration_s":0.042737,"scheduling_latency_s":0.042751} gitlab # [ 144.346618] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.310Z","message":"No existing merge request to be cleaned up."} gitlab # [ 144.347458] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.315Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f5714fab493c0d22df9a05ee","created_at":"2026-09-04T05:34:12.254Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-69ff20d0bbad4fec","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"22e657e9eb71d89cd4ab144c7d4b5f3f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.261Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f5714fab493c0d22df9a05ee: done: 0.010765 sec","job_status":"done","queue_duration_s":0.042737,"scheduling_latency_s":0.042751,"redis_calls":10,"redis_duration_s":0.001281,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000766,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007272,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010765,"completed_at":"2026-09-04T05:34:12.315Z","load_balancing_strategy":"primary","db_duration_s":0.000969,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.353594] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.316Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8c761bde572d16f23a647632","created_at":"2026-09-04T05:34:12.254Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-69ff20d0bbad4fec","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"22e657e9eb71d89cd4ab144c7d4b5f3f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.261Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8c761bde572d16f23a647632: start","job_status":"start","queue_duration_s":0.054312,"scheduling_latency_s":0.054326} gitlab # [ 144.364507] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.322Z","message":"No existing merge request to be cleaned up."} gitlab # [ 144.365918] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.326Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8c761bde572d16f23a647632","created_at":"2026-09-04T05:34:12.254Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-69ff20d0bbad4fec","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"22e657e9eb71d89cd4ab144c7d4b5f3f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.261Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8c761bde572d16f23a647632: done: 0.010573 sec","job_status":"done","queue_duration_s":0.054312,"scheduling_latency_s":0.054326,"redis_calls":10,"redis_duration_s":0.001339,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000594,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000745,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006916,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010573,"completed_at":"2026-09-04T05:34:12.326Z","load_balancing_strategy":"primary","db_duration_s":0.00097,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.405798] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.327Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"882c26f7938e87aa49377d3a","created_at":"2026-09-04T05:34:12.254Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-69ff20d0bbad4fec","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"22e657e9eb71d89cd4ab144c7d4b5f3f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.261Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-882c26f7938e87aa49377d3a: start","job_status":"start","queue_duration_s":0.065735,"scheduling_latency_s":0.065749} gitlab # [ 144.410160] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.334Z","message":"No existing merge request to be cleaned up."} gitlab # [ 144.411662] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.338Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"882c26f7938e87aa49377d3a","created_at":"2026-09-04T05:34:12.254Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-69ff20d0bbad4fec","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"22e657e9eb71d89cd4ab144c7d4b5f3f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.261Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-882c26f7938e87aa49377d3a: done: 0.011231 sec","job_status":"done","queue_duration_s":0.065735,"scheduling_latency_s":0.065749,"redis_calls":10,"redis_duration_s":0.001403,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000588,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000815,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007405,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011231,"completed_at":"2026-09-04T05:34:12.338Z","load_balancing_strategy":"primary","db_duration_s":0.000984,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.450706] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.384Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"132d564f65c6cceafc8a6c0d","created_at":"2026-09-04T05:34:12.184Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"188ac0f11882c55ed0a1ff6be5b444d3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.382Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-132d564f65c6cceafc8a6c0d: start","job_status":"start","queue_duration_s":0.001142,"scheduling_latency_s":0.001168} gitlab # [ 144.457732] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.400Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 144.460948] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.407Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"132d564f65c6cceafc8a6c0d","created_at":"2026-09-04T05:34:12.184Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"188ac0f11882c55ed0a1ff6be5b444d3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.382Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-132d564f65c6cceafc8a6c0d: done: 0.023641 sec","job_status":"done","queue_duration_s":0.001142,"scheduling_latency_s":0.001168,"redis_calls":10,"redis_duration_s":0.002271,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001157,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000859,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000255,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007662,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000213155,"duration_s":0.023641,"completed_at":"2026-09-04T05:34:12.407Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006179,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.499385] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.409Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"e3f5e2f6ce3ccd43c19dbbae","created_at":"2026-09-04T05:34:12.396Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"35f36822dd720e4c0ff076f9cc37bd38","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.406Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-e3f5e2f6ce3ccd43c19dbbae: start","job_status":"start","queue_duration_s":0.002892,"scheduling_latency_s":0.002905} gitlab # [ 144.541146] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.427Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 144.541912] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.437Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"e3f5e2f6ce3ccd43c19dbbae","created_at":"2026-09-04T05:34:12.396Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"35f36822dd720e4c0ff076f9cc37bd38","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.406Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-e3f5e2f6ce3ccd43c19dbbae: done: 0.027432 sec","job_status":"done","queue_duration_s":0.002892,"scheduling_latency_s":0.002905,"redis_calls":10,"redis_duration_s":0.005946,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000519,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004052,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001375,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008059,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001063543,"duration_s":0.027432,"completed_at":"2026-09-04T05:34:12.437Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001403,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.580994] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.440Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"c86fce1463380cb03aba4bd9","created_at":"2026-09-04T05:34:12.411Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"703a0089128090fa37c1156c6811c1f6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-04T05:34:12.420Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-c86fce1463380cb03aba4bd9: start","job_status":"start","queue_duration_s":0.019797,"scheduling_latency_s":0.019836} gitlab # [ 144.581684] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.457Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 144.582192] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.468Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"c86fce1463380cb03aba4bd9","created_at":"2026-09-04T05:34:12.411Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"703a0089128090fa37c1156c6811c1f6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-04T05:34:12.420Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-c86fce1463380cb03aba4bd9: done: 0.028491 sec","job_status":"done","queue_duration_s":0.019797,"scheduling_latency_s":0.019836,"redis_calls":10,"redis_duration_s":0.002374,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000258,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00156,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000556,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007363,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000476597,"duration_s":0.028491,"completed_at":"2026-09-04T05:34:12.468Z","load_balancing_strategy":"primary","db_duration_s":0.003107,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.623707] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.476Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"ff5c84bcdcb19ef7e9819d0a","created_at":"2026-09-04T05:34:12.427Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9a575b10d5d5957d24a2b1738e0c34a6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-04T05:34:12.436Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-ff5c84bcdcb19ef7e9819d0a: start","job_status":"start","queue_duration_s":0.040618,"scheduling_latency_s":0.040634} gitlab # [ 144.624575] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.540Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"ff5c84bcdcb19ef7e9819d0a","created_at":"2026-09-04T05:34:12.427Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9a575b10d5d5957d24a2b1738e0c34a6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-04T05:34:12.436Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-ff5c84bcdcb19ef7e9819d0a: done: 0.063411 sec","job_status":"done","queue_duration_s":0.040618,"scheduling_latency_s":0.040634,"redis_calls":10,"redis_duration_s":0.011141000000000002,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.005276,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005486,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000379,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015689,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001903873,"duration_s":0.063411,"completed_at":"2026-09-04T05:34:12.540Z","load_balancing_strategy":"primary","db_duration_s":0.022366,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.671349] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.545Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"df8a80714eb5da683f9ebb65","created_at":"2026-09-04T05:34:12.439Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7c7a7239576c9a545dc686e1dd778f6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.444Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-df8a80714eb5da683f9ebb65: start","job_status":"start","queue_duration_s":0.100745,"scheduling_latency_s":0.100763} gitlab # [ 144.672046] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.644Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"df8a80714eb5da683f9ebb65","created_at":"2026-09-04T05:34:12.439Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7c7a7239576c9a545dc686e1dd778f6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.444Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-df8a80714eb5da683f9ebb65: done: 0.099526 sec","job_status":"done","queue_duration_s":0.100745,"scheduling_latency_s":0.100763,"redis_calls":12,"redis_duration_s":0.017518,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000271,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.015251,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001996,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.036,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.036,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026157,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001357715,"exclusive_lock_hold_duration_s":0.037663725999999995,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.099526,"completed_at":"2026-09-04T05:34:12.644Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.049049,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.730921] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.646Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"88acd29200f413eb096be90d","created_at":"2026-09-04T05:34:12.446Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a1a852409a49e6fba962f306816208d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.458Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-88acd29200f413eb096be90d: start","job_status":"start","queue_duration_s":0.187523,"scheduling_latency_s":0.187538} gitlab # [ 144.731629] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.674Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"88acd29200f413eb096be90d","created_at":"2026-09-04T05:34:12.446Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a1a852409a49e6fba962f306816208d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.458Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-88acd29200f413eb096be90d: done: 0.028355 sec","job_status":"done","queue_duration_s":0.187523,"scheduling_latency_s":0.187538,"redis_calls":14,"redis_duration_s":0.001877,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000512,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000749,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000616,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014884,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000471568,"exclusive_lock_hold_duration_s":0.013432433,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.028355,"completed_at":"2026-09-04T05:34:12.674Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008414,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.772370] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.675Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"708dd1d6c381ef39af22bbad","created_at":"2026-09-04T05:34:12.466Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"43523cc563f6246e69f1bf918c48beed","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.471Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-708dd1d6c381ef39af22bbad: start","job_status":"start","queue_duration_s":0.203919,"scheduling_latency_s":0.203932} gitlab # [ 144.773059] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.901Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"708dd1d6c381ef39af22bbad","created_at":"2026-09-04T05:34:12.466Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"43523cc563f6246e69f1bf918c48beed","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.471Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-708dd1d6c381ef39af22bbad: done: 0.226273 sec","job_status":"done","queue_duration_s":0.203919,"scheduling_latency_s":0.203932,"redis_calls":565,"redis_duration_s":0.062149,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000251,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000896,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.061002,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.186576,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000469334,"duration_s":0.226273,"completed_at":"2026-09-04T05:34:12.901Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000244,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.817100] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.903Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"6c74a22f545fa7262545b1b5","created_at":"2026-09-04T05:34:12.476Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba135619a19a0d8a2e28ad291115bfce","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.480Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-6c74a22f545fa7262545b1b5: start","job_status":"start","queue_duration_s":0.422457,"scheduling_latency_s":0.422471} gitlab # [ 144.817831] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.925Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"6c74a22f545fa7262545b1b5","created_at":"2026-09-04T05:34:12.476Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba135619a19a0d8a2e28ad291115bfce","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.480Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-6c74a22f545fa7262545b1b5: done: 0.022285 sec","job_status":"done","queue_duration_s":0.422457,"scheduling_latency_s":0.422471,"redis_calls":16,"redis_duration_s":0.002092,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001017,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00083,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000245,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013302,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000336356,"duration_s":0.022285,"completed_at":"2026-09-04T05:34:12.925Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003166,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.867978] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.926Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"0a1fc9e3770ac139551ed09d","created_at":"2026-09-04T05:34:12.484Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b61703ce7b125f795a70e07f42384c63","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.492Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-0a1fc9e3770ac139551ed09d: start","job_status":"start","queue_duration_s":0.434518,"scheduling_latency_s":0.434531} gitlab # [ 144.868668] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.935Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 144.869183] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.939Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"0a1fc9e3770ac139551ed09d","created_at":"2026-09-04T05:34:12.484Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b61703ce7b125f795a70e07f42384c63","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.492Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-0a1fc9e3770ac139551ed09d: done: 0.013144 sec","job_status":"done","queue_duration_s":0.434518,"scheduling_latency_s":0.434531,"redis_calls":10,"redis_duration_s":0.001289,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000263,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000742,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000284,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007433,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000203936,"duration_s":0.013144,"completed_at":"2026-09-04T05:34:12.939Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000438,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.908052] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.940Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"b6d9693e9fc112be2954d9b1","created_at":"2026-09-04T05:34:12.508Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"06991c3b87eeb907823ca40fc655f8b1","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.511Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-b6d9693e9fc112be2954d9b1: start","job_status":"start","queue_duration_s":0.428948,"scheduling_latency_s":0.428961} gitlab # [ 144.915993] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.961Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"b6d9693e9fc112be2954d9b1","created_at":"2026-09-04T05:34:12.508Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"06991c3b87eeb907823ca40fc655f8b1","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.511Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-b6d9693e9fc112be2954d9b1: done: 0.021158 sec","job_status":"done","queue_duration_s":0.428948,"scheduling_latency_s":0.428961,"redis_calls":10,"redis_duration_s":0.001248,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000239,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00076,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000249,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00897,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000231594,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.021158,"completed_at":"2026-09-04T05:34:12.961Z","load_balancing_strategy":"primary","db_duration_s":0.007635,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.966964] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:12.962Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"f5ecbb5ab2709e44725efbb1","created_at":"2026-09-04T05:34:12.525Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.535Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-f5ecbb5ab2709e44725efbb1: start","job_status":"start","queue_duration_s":0.427505,"scheduling_latency_s":0.427518} gitlab # [ 144.967680] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.035Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"f5ecbb5ab2709e44725efbb1","created_at":"2026-09-04T05:34:12.525Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.535Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-f5ecbb5ab2709e44725efbb1: done: 0.072155 sec","job_status":"done","queue_duration_s":0.427505,"scheduling_latency_s":0.427518,"redis_calls":71,"redis_duration_s":0.016627,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002989,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.00057,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.00869,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.004378,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000381334,"duration_s":0.072155,"completed_at":"2026-09-04T05:34:13.035Z","load_balancing_strategy":"primary","db_duration_s":0.001778,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.009637] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.036Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"1fd6fdc6aa0c4df61b8b4492","created_at":"2026-09-04T05:34:12.544Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9adee9da6971c13d68f421f479fd8c75","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.553Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-1fd6fdc6aa0c4df61b8b4492: start","job_status":"start","queue_duration_s":0.482919,"scheduling_latency_s":0.482933} gitlab # [ 145.014987] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.050Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"1fd6fdc6aa0c4df61b8b4492","created_at":"2026-09-04T05:34:12.544Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9adee9da6971c13d68f421f479fd8c75","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.553Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-1fd6fdc6aa0c4df61b8b4492: done: 0.014148 sec","job_status":"done","queue_duration_s":0.482919,"scheduling_latency_s":0.482933,"redis_calls":12,"redis_duration_s":0.00181,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000241,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000875,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000694,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008144,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000295568,"exclusive_lock_hold_duration_s":0.001993549,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.014148,"completed_at":"2026-09-04T05:34:13.050Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001095,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.057113] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.051Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"8c45b116ed48aa909842e30e","created_at":"2026-09-04T05:34:12.562Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dc0cda7258114a7d66d4f3d0d9ef7db1","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.566Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-8c45b116ed48aa909842e30e: start","job_status":"start","queue_duration_s":0.485118,"scheduling_latency_s":0.485131} gitlab # [ 145.065961] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.069Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"8c45b116ed48aa909842e30e","created_at":"2026-09-04T05:34:12.562Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dc0cda7258114a7d66d4f3d0d9ef7db1","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.566Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-8c45b116ed48aa909842e30e: done: 0.018424 sec","job_status":"done","queue_duration_s":0.485118,"scheduling_latency_s":0.485131,"redis_calls":12,"redis_duration_s":0.001642,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000278,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000784,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00058,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000322667,"exclusive_lock_hold_duration_s":0.003308522,"duration_s":0.018424,"completed_at":"2026-09-04T05:34:13.069Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00453,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.122760] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.070Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"9f7f57e152d115c66ceb2106","created_at":"2026-09-04T05:34:12.575Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"058ad72b0a967da48a6194b84c77322b","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.580Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-9f7f57e152d115c66ceb2106: start","job_status":"start","queue_duration_s":0.49069,"scheduling_latency_s":0.490712} gitlab # [ 145.123593] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.086Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"9f7f57e152d115c66ceb2106","created_at":"2026-09-04T05:34:12.575Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"058ad72b0a967da48a6194b84c77322b","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.580Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-9f7f57e152d115c66ceb2106: done: 0.016013 sec","job_status":"done","queue_duration_s":0.49069,"scheduling_latency_s":0.490712,"redis_calls":10,"redis_duration_s":0.001332,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00026,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000804,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000268,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000207568,"duration_s":0.016013,"completed_at":"2026-09-04T05:34:13.086Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002218,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.180235] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.087Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"18fb56aa10ecd0f106027a63","created_at":"2026-09-04T05:34:12.585Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4fa253e1ab8deac583455eb380cb1f76","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.598Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-18fb56aa10ecd0f106027a63: start","job_status":"start","queue_duration_s":0.48914,"scheduling_latency_s":0.489153} gitlab # [ 145.180983] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.099Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"18fb56aa10ecd0f106027a63","created_at":"2026-09-04T05:34:12.585Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4fa253e1ab8deac583455eb380cb1f76","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.598Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-18fb56aa10ecd0f106027a63: done: 0.011479 sec","job_status":"done","queue_duration_s":0.48914,"scheduling_latency_s":0.489153,"redis_calls":10,"redis_duration_s":0.001206,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000239,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000727,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00024,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006855,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000187733,"duration_s":0.011479,"completed_at":"2026-09-04T05:34:13.099Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000246,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.227653] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.100Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"c0def1913a141781a80bd655","created_at":"2026-09-04T05:34:12.602Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8ddc37521b7c1791cf5479a95daaa301","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.606Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-c0def1913a141781a80bd655: start","job_status":"start","queue_duration_s":0.493501,"scheduling_latency_s":0.493514} gitlab # [ 145.228549] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.238Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"c0def1913a141781a80bd655","created_at":"2026-09-04T05:34:12.602Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8ddc37521b7c1791cf5479a95daaa301","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.606Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-c0def1913a141781a80bd655: done: 0.138853 sec","job_status":"done","queue_duration_s":0.493501,"scheduling_latency_s":0.493514,"redis_calls":14,"redis_duration_s":0.0020540000000000003,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000508,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00084,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000706,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.097623,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000319315,"exclusive_lock_hold_duration_s":0.124693781,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.138853,"completed_at":"2026-09-04T05:34:13.238Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.04446,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.254676] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.240Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"57efff63fb8df25afe7f580e","created_at":"2026-09-04T05:34:12.610Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5997f5ecd39fe161815e7a6bd76b62d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.615Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-57efff63fb8df25afe7f580e: start","job_status":"start","queue_duration_s":0.624646,"scheduling_latency_s":0.624661} gitlab # [ 145.293654] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.256Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"57efff63fb8df25afe7f580e","created_at":"2026-09-04T05:34:12.610Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5997f5ecd39fe161815e7a6bd76b62d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.615Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-57efff63fb8df25afe7f580e: done: 0.016485 sec","job_status":"done","queue_duration_s":0.624646,"scheduling_latency_s":0.624661,"redis_calls":12,"redis_duration_s":0.001577,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000252,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000766,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000559,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009367,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000297803,"exclusive_lock_hold_duration_s":0.00399129,"duration_s":0.016485,"completed_at":"2026-09-04T05:34:13.256Z","load_balancing_strategy":"primary","db_duration_s":0.001836,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.302209] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.257Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"99cf21989c97d024eee46959","created_at":"2026-09-04T05:34:12.618Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"45cfc6ba3f2b522b0d72d31ce21c86ec","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.623Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-99cf21989c97d024eee46959: start","job_status":"start","queue_duration_s":0.633985,"scheduling_latency_s":0.633997} gitlab # [ 145.315537] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.264Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 145.319096] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.268Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"99cf21989c97d024eee46959","created_at":"2026-09-04T05:34:12.618Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-4a2c4b5b98f345d3","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"45cfc6ba3f2b522b0d72d31ce21c86ec","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:12.623Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-99cf21989c97d024eee46959: done: 0.011473 sec","job_status":"done","queue_duration_s":0.633985,"scheduling_latency_s":0.633997,"redis_calls":10,"redis_duration_s":0.001344,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000241,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000852,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000251,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006747,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000194438,"duration_s":0.011473,"completed_at":"2026-09-04T05:34:13.268Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000233,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.363752] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.269Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54c73f49b28ea12f22c2f36c","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54c73f49b28ea12f22c2f36c: start","job_status":"start","queue_duration_s":0.241199,"scheduling_latency_s":0.241212} gitlab # [ 145.367078] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.286Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54c73f49b28ea12f22c2f36c","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54c73f49b28ea12f22c2f36c: done: 0.016416 sec","job_status":"done","queue_duration_s":0.241199,"scheduling_latency_s":0.241212,"redis_calls":16,"redis_duration_s":0.002192,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000253,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000649,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00129,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008222,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000191365,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016416,"completed_at":"2026-09-04T05:34:13.286Z","load_balancing_strategy":"primary","db_duration_s":0.002811,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.407128] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.287Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f05283fed64cb590c5c9beb","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f05283fed64cb590c5c9beb: start","job_status":"start","queue_duration_s":0.25846,"scheduling_latency_s":0.258473} gitlab # [ 145.413825] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.297Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f05283fed64cb590c5c9beb","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f05283fed64cb590c5c9beb: done: 0.009855 sec","job_status":"done","queue_duration_s":0.25846,"scheduling_latency_s":0.258473,"redis_calls":13,"redis_duration_s":0.0015769999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001076,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006291,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009855,"completed_at":"2026-09-04T05:34:13.297Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.456586] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.297Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9726e26a9960a6e46b6b9c8","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9726e26a9960a6e46b6b9c8: start","job_status":"start","queue_duration_s":0.269165,"scheduling_latency_s":0.269178} gitlab # [ 145.458865] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.307Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9726e26a9960a6e46b6b9c8","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9726e26a9960a6e46b6b9c8: done: 0.009227 sec","job_status":"done","queue_duration_s":0.269165,"scheduling_latency_s":0.269178,"redis_calls":13,"redis_duration_s":0.001507,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0005,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001007,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006162,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009227,"completed_at":"2026-09-04T05:34:13.307Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.515973] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.308Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5ef0f18e80815aa3530626ae","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5ef0f18e80815aa3530626ae: start","job_status":"start","queue_duration_s":0.279359,"scheduling_latency_s":0.279371} gitlab # [ 145.516666] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.317Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5ef0f18e80815aa3530626ae","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5ef0f18e80815aa3530626ae: done: 0.009244 sec","job_status":"done","queue_duration_s":0.279359,"scheduling_latency_s":0.279371,"redis_calls":13,"redis_duration_s":0.0015,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000463,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001037,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006148,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009244,"completed_at":"2026-09-04T05:34:13.317Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.560731] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.318Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f82960a3d702a238069ee6bd","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f82960a3d702a238069ee6bd: start","job_status":"start","queue_duration_s":0.289554,"scheduling_latency_s":0.289567} gitlab # [ 145.561484] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.327Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f82960a3d702a238069ee6bd","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f82960a3d702a238069ee6bd: done: 0.009441 sec","job_status":"done","queue_duration_s":0.289554,"scheduling_latency_s":0.289567,"redis_calls":13,"redis_duration_s":0.001462,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000482,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00098,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006393,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009441,"completed_at":"2026-09-04T05:34:13.327Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.617192] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.328Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"10c11957558e2a5453a1f64f","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-10c11957558e2a5453a1f64f: start","job_status":"start","queue_duration_s":0.299927,"scheduling_latency_s":0.299939} gitlab # [ 145.617934] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.338Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"10c11957558e2a5453a1f64f","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-10c11957558e2a5453a1f64f: done: 0.009557 sec","job_status":"done","queue_duration_s":0.299927,"scheduling_latency_s":0.299939,"redis_calls":13,"redis_duration_s":0.001669,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000636,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006151,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009557,"completed_at":"2026-09-04T05:34:13.338Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.630585] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.339Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19e0a7d79ea06890180b4265","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19e0a7d79ea06890180b4265: start","job_status":"start","queue_duration_s":0.310445,"scheduling_latency_s":0.310457} gitlab # [ 145.668324] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.348Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19e0a7d79ea06890180b4265","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19e0a7d79ea06890180b4265: done: 0.009394 sec","job_status":"done","queue_duration_s":0.310445,"scheduling_latency_s":0.310457,"redis_calls":13,"redis_duration_s":0.001482,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000484,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000998,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006413,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009394,"completed_at":"2026-09-04T05:34:13.348Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.709874] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.349Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8fd4e564a172a418095f40c7","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8fd4e564a172a418095f40c7: start","job_status":"start","queue_duration_s":0.320792,"scheduling_latency_s":0.320806} gitlab # [ 145.710737] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.358Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8fd4e564a172a418095f40c7","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8fd4e564a172a418095f40c7: done: 0.009259 sec","job_status":"done","queue_duration_s":0.320792,"scheduling_latency_s":0.320806,"redis_calls":13,"redis_duration_s":0.001597,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001073,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006261,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009259,"completed_at":"2026-09-04T05:34:13.358Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.757193] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.359Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fbf56fe242500c1819569f0a","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fbf56fe242500c1819569f0a: start","job_status":"start","queue_duration_s":0.331162,"scheduling_latency_s":0.331176} gitlab # [ 145.757934] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.369Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fbf56fe242500c1819569f0a","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fbf56fe242500c1819569f0a: done: 0.009256 sec","job_status":"done","queue_duration_s":0.331162,"scheduling_latency_s":0.331176,"redis_calls":13,"redis_duration_s":0.001506,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006214,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009256,"completed_at":"2026-09-04T05:34:13.368Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.799315] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.369Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad3218ebc6249b937be0cbab","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad3218ebc6249b937be0cbab: start","job_status":"start","queue_duration_s":0.341104,"scheduling_latency_s":0.341118} gitlab # [ 145.799913] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.379Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad3218ebc6249b937be0cbab","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad3218ebc6249b937be0cbab: done: 0.009393 sec","job_status":"done","queue_duration_s":0.341104,"scheduling_latency_s":0.341118,"redis_calls":13,"redis_duration_s":0.001646,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001119,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006254,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009393,"completed_at":"2026-09-04T05:34:13.379Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.840240] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.380Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03d7974bad3ff74105d2a76d","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03d7974bad3ff74105d2a76d: start","job_status":"start","queue_duration_s":0.351325,"scheduling_latency_s":0.351338} gitlab # [ 145.840924] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.389Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03d7974bad3ff74105d2a76d","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03d7974bad3ff74105d2a76d: done: 0.00959 sec","job_status":"done","queue_duration_s":0.351325,"scheduling_latency_s":0.351338,"redis_calls":13,"redis_duration_s":0.001546,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00101,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006059,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00959,"completed_at":"2026-09-04T05:34:13.389Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.882437] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.390Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4a438c071f1f9f4c24a0af2f","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4a438c071f1f9f4c24a0af2f: start","job_status":"start","queue_duration_s":0.361734,"scheduling_latency_s":0.361748} gitlab # [ 145.883140] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.399Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4a438c071f1f9f4c24a0af2f","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4a438c071f1f9f4c24a0af2f: done: 0.009553 sec","job_status":"done","queue_duration_s":0.361734,"scheduling_latency_s":0.361748,"redis_calls":13,"redis_duration_s":0.0015240000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001015,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005947,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009553,"completed_at":"2026-09-04T05:34:13.399Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.934090] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.400Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"821fa089d1dc5924b3aee08e","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-821fa089d1dc5924b3aee08e: start","job_status":"start","queue_duration_s":0.372082,"scheduling_latency_s":0.372095} gitlab # [ 145.934818] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.410Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"821fa089d1dc5924b3aee08e","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-821fa089d1dc5924b3aee08e: done: 0.009395 sec","job_status":"done","queue_duration_s":0.372082,"scheduling_latency_s":0.372095,"redis_calls":13,"redis_duration_s":0.0015530000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001036,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005932,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009395,"completed_at":"2026-09-04T05:34:13.410Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.944914] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.411Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"828950a2e91893bbd7e83ca0","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-828950a2e91893bbd7e83ca0: start","job_status":"start","queue_duration_s":0.3823,"scheduling_latency_s":0.382312} gitlab # [ 145.989726] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.420Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"828950a2e91893bbd7e83ca0","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-828950a2e91893bbd7e83ca0: done: 0.009289 sec","job_status":"done","queue_duration_s":0.3823,"scheduling_latency_s":0.382312,"redis_calls":13,"redis_duration_s":0.001491,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000493,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000998,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005955,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009289,"completed_at":"2026-09-04T05:34:13.420Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.991181] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.421Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0798815125894528e7a8b925","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0798815125894528e7a8b925: start","job_status":"start","queue_duration_s":0.392532,"scheduling_latency_s":0.392544} gitlab # [ 146.029206] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.430Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0798815125894528e7a8b925","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0798815125894528e7a8b925: done: 0.009655 sec","job_status":"done","queue_duration_s":0.392532,"scheduling_latency_s":0.392544,"redis_calls":13,"redis_duration_s":0.001498,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000988,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005809,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009655,"completed_at":"2026-09-04T05:34:13.430Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.034981] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.431Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d3338428a6a6b3fe80f1b61b","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d3338428a6a6b3fe80f1b61b: start","job_status":"start","queue_duration_s":0.403189,"scheduling_latency_s":0.403202} gitlab # [ 146.045650] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.441Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d3338428a6a6b3fe80f1b61b","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d3338428a6a6b3fe80f1b61b: done: 0.009396 sec","job_status":"done","queue_duration_s":0.403189,"scheduling_latency_s":0.403202,"redis_calls":13,"redis_duration_s":0.001613,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001095,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006202,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009396,"completed_at":"2026-09-04T05:34:13.441Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.086140] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.442Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16a9e7ac1d3727df3cf31aa6","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16a9e7ac1d3727df3cf31aa6: start","job_status":"start","queue_duration_s":0.413594,"scheduling_latency_s":0.413607} gitlab # [ 146.088839] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.452Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16a9e7ac1d3727df3cf31aa6","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16a9e7ac1d3727df3cf31aa6: done: 0.009706 sec","job_status":"done","queue_duration_s":0.413594,"scheduling_latency_s":0.413607,"redis_calls":13,"redis_duration_s":0.0015760000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006412,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009706,"completed_at":"2026-09-04T05:34:13.451Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.127704] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.453Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f4f270ed57b6476eb559196","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f4f270ed57b6476eb559196: start","job_status":"start","queue_duration_s":0.424361,"scheduling_latency_s":0.424374} gitlab # [ 146.166742] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.464Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f4f270ed57b6476eb559196","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f4f270ed57b6476eb559196: done: 0.011885 sec","job_status":"done","queue_duration_s":0.424361,"scheduling_latency_s":0.424374,"redis_calls":13,"redis_duration_s":0.0019950000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001466,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008227,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011885,"completed_at":"2026-09-04T05:34:13.464Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.168447] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.466Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26be522d161d9cb4df52c84d","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26be522d161d9cb4df52c84d: start","job_status":"start","queue_duration_s":0.43736,"scheduling_latency_s":0.43738} gitlab # [ 146.211986] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.522Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26be522d161d9cb4df52c84d","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26be522d161d9cb4df52c84d: done: 0.056701 sec","job_status":"done","queue_duration_s":0.43736,"scheduling_latency_s":0.43738,"redis_calls":13,"redis_duration_s":0.001488,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000998,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.052173,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.056701,"completed_at":"2026-09-04T05:34:13.522Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.213083] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.523Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"750000643a96674397fcc971","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-750000643a96674397fcc971: start","job_status":"start","queue_duration_s":0.495089,"scheduling_latency_s":0.495106} gitlab # [ 146.224523] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.537Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"750000643a96674397fcc971","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-750000643a96674397fcc971: done: 0.013206 sec","job_status":"done","queue_duration_s":0.495089,"scheduling_latency_s":0.495106,"redis_calls":13,"redis_duration_s":0.002003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000561,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001442,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013206,"completed_at":"2026-09-04T05:34:13.536Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.274618] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.538Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7df3a34a347de1c9ab46e226","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7df3a34a347de1c9ab46e226: start","job_status":"start","queue_duration_s":0.509385,"scheduling_latency_s":0.509406} gitlab # [ 146.275593] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.548Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7df3a34a347de1c9ab46e226","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7df3a34a347de1c9ab46e226: done: 0.010529 sec","job_status":"done","queue_duration_s":0.509385,"scheduling_latency_s":0.509406,"redis_calls":13,"redis_duration_s":0.001793,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000662,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001131,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006629,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010529,"completed_at":"2026-09-04T05:34:13.548Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.329750] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.549Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fffacbc3d1d0f19714739bc6","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fffacbc3d1d0f19714739bc6: start","job_status":"start","queue_duration_s":0.52082,"scheduling_latency_s":0.520835} gitlab # [ 146.330574] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.559Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fffacbc3d1d0f19714739bc6","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fffacbc3d1d0f19714739bc6: done: 0.009577 sec","job_status":"done","queue_duration_s":0.52082,"scheduling_latency_s":0.520835,"redis_calls":13,"redis_duration_s":0.001434,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000475,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000959,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006042,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009577,"completed_at":"2026-09-04T05:34:13.559Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.348702] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.560Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b01c4991c3526449767e0be","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b01c4991c3526449767e0be: start","job_status":"start","queue_duration_s":0.531524,"scheduling_latency_s":0.531538} gitlab # [ 146.359721] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.570Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b01c4991c3526449767e0be","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b01c4991c3526449767e0be: done: 0.009909 sec","job_status":"done","queue_duration_s":0.531524,"scheduling_latency_s":0.531538,"redis_calls":13,"redis_duration_s":0.001611,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000593,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001018,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009909,"completed_at":"2026-09-04T05:34:13.570Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.406737] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.571Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d72172aae3cc47ef28ad9af","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d72172aae3cc47ef28ad9af: start","job_status":"start","queue_duration_s":0.542371,"scheduling_latency_s":0.542387} gitlab # [ 146.407470] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.580Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d72172aae3cc47ef28ad9af","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d72172aae3cc47ef28ad9af: done: 0.009362 sec","job_status":"done","queue_duration_s":0.542371,"scheduling_latency_s":0.542387,"redis_calls":13,"redis_duration_s":0.001881,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000476,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001405,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006243,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009362,"completed_at":"2026-09-04T05:34:13.580Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.440321] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.581Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6788393af8b1c9e5d1220158","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6788393af8b1c9e5d1220158: start","job_status":"start","queue_duration_s":0.55252,"scheduling_latency_s":0.552532} gitlab # [ 146.451497] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.590Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6788393af8b1c9e5d1220158","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6788393af8b1c9e5d1220158: done: 0.008931 sec","job_status":"done","queue_duration_s":0.55252,"scheduling_latency_s":0.552532,"redis_calls":13,"redis_duration_s":0.001437,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000468,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000969,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006074,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008931,"completed_at":"2026-09-04T05:34:13.590Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.505051] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.590Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e91574a83043100295a8c048","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e91574a83043100295a8c048: start","job_status":"start","queue_duration_s":0.562243,"scheduling_latency_s":0.562255} gitlab # [ 146.505797] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.599Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e91574a83043100295a8c048","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e91574a83043100295a8c048: done: 0.008921 sec","job_status":"done","queue_duration_s":0.562243,"scheduling_latency_s":0.562255,"redis_calls":13,"redis_duration_s":0.001447,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000465,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000982,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005659,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008921,"completed_at":"2026-09-04T05:34:13.599Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.548714] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.600Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"981c2c6d6ab02a209869172a","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-981c2c6d6ab02a209869172a: start","job_status":"start","queue_duration_s":0.572209,"scheduling_latency_s":0.572222} gitlab # [ 146.549649] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.609Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"981c2c6d6ab02a209869172a","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-981c2c6d6ab02a209869172a: done: 0.009062 sec","job_status":"done","queue_duration_s":0.572209,"scheduling_latency_s":0.572222,"redis_calls":13,"redis_duration_s":0.001604,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001114,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005871,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009062,"completed_at":"2026-09-04T05:34:13.609Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.594671] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.610Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"513b7c89ad96844630b7d01c","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-513b7c89ad96844630b7d01c: start","job_status":"start","queue_duration_s":0.582095,"scheduling_latency_s":0.582107} gitlab # [ 146.595445] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.619Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"513b7c89ad96844630b7d01c","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-513b7c89ad96844630b7d01c: done: 0.008879 sec","job_status":"done","queue_duration_s":0.582095,"scheduling_latency_s":0.582107,"redis_calls":13,"redis_duration_s":0.00151,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000967,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00576,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008879,"completed_at":"2026-09-04T05:34:13.619Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.649878] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.620Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d008a43bb16b21caca67fec","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d008a43bb16b21caca67fec: start","job_status":"start","queue_duration_s":0.591843,"scheduling_latency_s":0.591855} gitlab # [ 146.650594] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.629Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d008a43bb16b21caca67fec","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d008a43bb16b21caca67fec: done: 0.009006 sec","job_status":"done","queue_duration_s":0.591843,"scheduling_latency_s":0.591855,"redis_calls":13,"redis_duration_s":0.00154,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001001,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00587,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009006,"completed_at":"2026-09-04T05:34:13.629Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.690945] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.630Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5daa8b7a428845ce9fe5c5cd","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5daa8b7a428845ce9fe5c5cd: start","job_status":"start","queue_duration_s":0.601646,"scheduling_latency_s":0.60166} gitlab # [ 146.691683] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.639Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5daa8b7a428845ce9fe5c5cd","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5daa8b7a428845ce9fe5c5cd: done: 0.009235 sec","job_status":"done","queue_duration_s":0.601646,"scheduling_latency_s":0.60166,"redis_calls":13,"redis_duration_s":0.00143,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00047,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005646,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009235,"completed_at":"2026-09-04T05:34:13.639Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.711610] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.640Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f64adc9acda21bb1cca615b1","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f64adc9acda21bb1cca615b1: start","job_status":"start","queue_duration_s":0.61165,"scheduling_latency_s":0.611663} gitlab # [ 146.756129] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.649Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f64adc9acda21bb1cca615b1","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f64adc9acda21bb1cca615b1: done: 0.009003 sec","job_status":"done","queue_duration_s":0.61165,"scheduling_latency_s":0.611663,"redis_calls":13,"redis_duration_s":0.001486,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00048,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001006,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009003,"completed_at":"2026-09-04T05:34:13.649Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.756886] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.650Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e4fdf2de2da1385767c4f94","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e4fdf2de2da1385767c4f94: start","job_status":"start","queue_duration_s":0.621456,"scheduling_latency_s":0.621469} gitlab # [ 146.767224] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.659Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e4fdf2de2da1385767c4f94","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e4fdf2de2da1385767c4f94: done: 0.008942 sec","job_status":"done","queue_duration_s":0.621456,"scheduling_latency_s":0.621469,"redis_calls":13,"redis_duration_s":0.001863,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000845,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001018,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005964,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008942,"completed_at":"2026-09-04T05:34:13.659Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.809743] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.659Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6b9fadb55b6e6a7a00c9ce6","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6b9fadb55b6e6a7a00c9ce6: start","job_status":"start","queue_duration_s":0.631191,"scheduling_latency_s":0.631203} gitlab # [ 146.848393] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.668Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6b9fadb55b6e6a7a00c9ce6","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6b9fadb55b6e6a7a00c9ce6: done: 0.008957 sec","job_status":"done","queue_duration_s":0.631191,"scheduling_latency_s":0.631203,"redis_calls":13,"redis_duration_s":0.001585,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001072,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005529,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008957,"completed_at":"2026-09-04T05:34:13.668Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.849523] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.670Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1364ade2f3fd701d9e7ac03d","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1364ade2f3fd701d9e7ac03d: start","job_status":"start","queue_duration_s":0.641266,"scheduling_latency_s":0.641279} gitlab # [ 146.889464] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.678Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1364ade2f3fd701d9e7ac03d","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1364ade2f3fd701d9e7ac03d: done: 0.008861 sec","job_status":"done","queue_duration_s":0.641266,"scheduling_latency_s":0.641279,"redis_calls":13,"redis_duration_s":0.0015249999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000491,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001034,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005959,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008861,"completed_at":"2026-09-04T05:34:13.678Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.891629] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.679Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa56f735fb15d00ecb03a780","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa56f735fb15d00ecb03a780: start","job_status":"start","queue_duration_s":0.650908,"scheduling_latency_s":0.650921} gitlab # [ 146.938526] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.688Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa56f735fb15d00ecb03a780","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa56f735fb15d00ecb03a780: done: 0.008819 sec","job_status":"done","queue_duration_s":0.650908,"scheduling_latency_s":0.650921,"redis_calls":13,"redis_duration_s":0.001425,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000479,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000946,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006009,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008819,"completed_at":"2026-09-04T05:34:13.688Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.939566] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.689Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf5e86ccdc127c711ee74a99","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf5e86ccdc127c711ee74a99: start","job_status":"start","queue_duration_s":0.660514,"scheduling_latency_s":0.660526} gitlab # [ 146.947943] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.698Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf5e86ccdc127c711ee74a99","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf5e86ccdc127c711ee74a99: done: 0.008784 sec","job_status":"done","queue_duration_s":0.660514,"scheduling_latency_s":0.660526,"redis_calls":13,"redis_duration_s":0.001447,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000483,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005913,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008784,"completed_at":"2026-09-04T05:34:13.697Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 146.990659] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.698Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f224d3ece3defbc8dc95080","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f224d3ece3defbc8dc95080: start","job_status":"start","queue_duration_s":0.670165,"scheduling_latency_s":0.670177} gitlab # [ 146.992751] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.707Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f224d3ece3defbc8dc95080","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f224d3ece3defbc8dc95080: done: 0.008891 sec","job_status":"done","queue_duration_s":0.670165,"scheduling_latency_s":0.670177,"redis_calls":13,"redis_duration_s":0.001457,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000487,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00097,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005601,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008891,"completed_at":"2026-09-04T05:34:13.707Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.049294] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.709Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78564f7979c4886bf0cba2ba","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78564f7979c4886bf0cba2ba: start","job_status":"start","queue_duration_s":0.680269,"scheduling_latency_s":0.680283} gitlab # [ 147.049907] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.718Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78564f7979c4886bf0cba2ba","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78564f7979c4886bf0cba2ba: done: 0.009176 sec","job_status":"done","queue_duration_s":0.680269,"scheduling_latency_s":0.680283,"redis_calls":13,"redis_duration_s":0.001657,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001118,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005872,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009176,"completed_at":"2026-09-04T05:34:13.718Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.095705] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.719Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de4e3c801397b636e787ea4c","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de4e3c801397b636e787ea4c: start","job_status":"start","queue_duration_s":0.690265,"scheduling_latency_s":0.690278} gitlab # [ 147.096545] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.727Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de4e3c801397b636e787ea4c","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de4e3c801397b636e787ea4c: done: 0.009021 sec","job_status":"done","queue_duration_s":0.690265,"scheduling_latency_s":0.690278,"redis_calls":13,"redis_duration_s":0.0018700000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000491,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001379,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00584,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009021,"completed_at":"2026-09-04T05:34:13.727Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.118751] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.728Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43efe6c6c5815e9dded0604e","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43efe6c6c5815e9dded0604e: start","job_status":"start","queue_duration_s":0.700177,"scheduling_latency_s":0.70019} gitlab # [ 147.158358] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.737Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43efe6c6c5815e9dded0604e","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43efe6c6c5815e9dded0604e: done: 0.009089 sec","job_status":"done","queue_duration_s":0.700177,"scheduling_latency_s":0.70019,"redis_calls":13,"redis_duration_s":0.0016539999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000493,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001161,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006016,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009089,"completed_at":"2026-09-04T05:34:13.737Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.163841] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.738Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4aac5a098e39306f7d462220","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4aac5a098e39306f7d462220: start","job_status":"start","queue_duration_s":0.710076,"scheduling_latency_s":0.710089} gitlab # [ 147.174932] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.747Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4aac5a098e39306f7d462220","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4aac5a098e39306f7d462220: done: 0.008828 sec","job_status":"done","queue_duration_s":0.710076,"scheduling_latency_s":0.710089,"redis_calls":13,"redis_duration_s":0.001487,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000472,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001015,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005648,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008828,"completed_at":"2026-09-04T05:34:13.747Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.213702] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.748Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9d43f190520189b382ce139","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9d43f190520189b382ce139: start","job_status":"start","queue_duration_s":0.719936,"scheduling_latency_s":0.719949} gitlab # [ 147.254961] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.757Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9d43f190520189b382ce139","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9d43f190520189b382ce139: done: 0.008838 sec","job_status":"done","queue_duration_s":0.719936,"scheduling_latency_s":0.719949,"redis_calls":13,"redis_duration_s":0.00178,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000806,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000974,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00595,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008838,"completed_at":"2026-09-04T05:34:13.757Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.255943] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.758Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15766750853d1b5b36ef4491","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15766750853d1b5b36ef4491: start","job_status":"start","queue_duration_s":0.729545,"scheduling_latency_s":0.729557} gitlab # [ 147.264477] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.766Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15766750853d1b5b36ef4491","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15766750853d1b5b36ef4491: done: 0.008732 sec","job_status":"done","queue_duration_s":0.729545,"scheduling_latency_s":0.729557,"redis_calls":13,"redis_duration_s":0.0014810000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000979,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005896,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008732,"completed_at":"2026-09-04T05:34:13.766Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.311733] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.767Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31a91481a59a1249635102a8","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31a91481a59a1249635102a8: start","job_status":"start","queue_duration_s":0.739145,"scheduling_latency_s":0.739158} gitlab # [ 147.312920] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.777Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31a91481a59a1249635102a8","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31a91481a59a1249635102a8: done: 0.009619 sec","job_status":"done","queue_duration_s":0.739145,"scheduling_latency_s":0.739158,"redis_calls":13,"redis_duration_s":0.00151,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000489,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006077,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009619,"completed_at":"2026-09-04T05:34:13.777Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.352486] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.778Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a543419042bd35524fe7c640","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a543419042bd35524fe7c640: start","job_status":"start","queue_duration_s":0.749616,"scheduling_latency_s":0.749628} gitlab # [ 147.391025] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.787Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a543419042bd35524fe7c640","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a543419042bd35524fe7c640: done: 0.009193 sec","job_status":"done","queue_duration_s":0.749616,"scheduling_latency_s":0.749628,"redis_calls":13,"redis_duration_s":0.001498,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005755,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009193,"completed_at":"2026-09-04T05:34:13.787Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.392059] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.788Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc07a939eb0bd8b8d3e523e6","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc07a939eb0bd8b8d3e523e6: start","job_status":"start","queue_duration_s":0.760169,"scheduling_latency_s":0.760183} gitlab # [ 147.400694] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.797Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc07a939eb0bd8b8d3e523e6","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc07a939eb0bd8b8d3e523e6: done: 0.009073 sec","job_status":"done","queue_duration_s":0.760169,"scheduling_latency_s":0.760183,"redis_calls":13,"redis_duration_s":0.001497,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000993,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009073,"completed_at":"2026-09-04T05:34:13.797Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.449032] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.798Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0202950650faec720276197d","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0202950650faec720276197d: start","job_status":"start","queue_duration_s":0.770046,"scheduling_latency_s":0.77006} gitlab # [ 147.449802] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0202950650faec720276197d","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0202950650faec720276197d: done: 0.009056 sec","job_status":"done","queue_duration_s":0.770046,"scheduling_latency_s":0.77006,"redis_calls":13,"redis_duration_s":0.001428,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000486,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000942,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006168,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009056,"completed_at":"2026-09-04T05:34:13.807Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.493639] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.808Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7d9195b329a663d63a64960","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7d9195b329a663d63a64960: start","job_status":"start","queue_duration_s":0.779897,"scheduling_latency_s":0.77991} gitlab # [ 147.494286] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.817Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7d9195b329a663d63a64960","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7d9195b329a663d63a64960: done: 0.008926 sec","job_status":"done","queue_duration_s":0.779897,"scheduling_latency_s":0.77991,"redis_calls":13,"redis_duration_s":0.001472,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000484,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000988,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006046,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008926,"completed_at":"2026-09-04T05:34:13.817Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.541525] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.818Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"edbba3cd8785e2744e49f760","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-edbba3cd8785e2744e49f760: start","job_status":"start","queue_duration_s":0.789648,"scheduling_latency_s":0.789661} gitlab # [ 147.542247] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.827Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"edbba3cd8785e2744e49f760","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-edbba3cd8785e2744e49f760: done: 0.009221 sec","job_status":"done","queue_duration_s":0.789648,"scheduling_latency_s":0.789661,"redis_calls":13,"redis_duration_s":0.0014949999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000478,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00558,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009221,"completed_at":"2026-09-04T05:34:13.827Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.606092] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.828Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e8dc74963b3249fb0dd7b68","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e8dc74963b3249fb0dd7b68: start","job_status":"start","queue_duration_s":0.799735,"scheduling_latency_s":0.799748} gitlab # [ 147.606832] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:13.837Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e8dc74963b3249fb0dd7b68","created_at":"2026-09-04T05:34:12.982Z","trace_propagation_headers":{"sentry-trace":"2e7fb18b15a0470aae49d3e7ec7b68b2-c799310eadf7438c","baggage":"sentry-trace_id=2e7fb18b15a0470aae49d3e7ec7b68b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f38c059278776d39a07998abda813a6b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:13.028Z","job_size_bytes":2,"pid":1611,"sidekiq_tid":"maeb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e8dc74963b3249fb0dd7b68: done: 0.009122 sec","job_status":"done","queue_duration_s":0.799735,"scheduling_latency_s":0.799748,"redis_calls":13,"redis_duration_s":0.001613,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0005,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001113,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00607,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009122,"completed_at":"2026-09-04T05:34:13.837Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.790192] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 3387008760 wd_nsec: 3387008567 gitlab # [ 158.829805] setup-gitlab-start[2212]: W, [2026-09-04T05:34:26.978391 #2212] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 180.287790] postgres[2408]: [2408] ERROR: relation "table_without_models" does not exist gitlab # [ 180.290865] postgres[2408]: [2408] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 183.724251] runuser[2211]: pam_unix(runuser:session): session closed for user gitlab gitlab # [ 183.727757] setup-gitlab-start[2189]: + systemctl restart gitlab-sidekiq.service gitlab # [ 183.743288] systemd[1]: Stopping gitlab-sidekiq.service... gitlab # [ 183.744765] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:51.892Z","message":"Shutting down"} gitlab # [ 183.745803] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:51.892Z","message":"Scheduler exiting..."} gitlab # [ 183.746942] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:51.894Z","message":"Terminating quiet threads for default capsule"} gitlab # [ 183.748060] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:51.894Z","message":"Scheduler exiting..."} gitlab # [ 184.246830] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:52.395Z","message":"Pausing to allow jobs to finish..."} gitlab # [ 186.250996] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:54.399Z","message":"Bye!"} gitlab # [ 186.275688] sidekiq-cluster[1611]: {"severity":"INFO","time":"2026-09-04T05:34:54.424Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1611,"worker_id":"sidekiq_0","memwd_rss_bytes":938258432,"retry":0} gitlab # [ 187.322848] systemd[1]: gitlab-sidekiq.service: Deactivated successfully. gitlab # [ 187.324531] systemd[1]: Stopped gitlab-sidekiq.service. gitlab # [ 187.326643] systemd[1]: gitlab-sidekiq.service: Consumed 54.628s CPU time over 1min 55.062s wall clock time, 957.9M memory peak, 4.4M read from disk. gitlab # [ 187.332158] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 187.334841] setup-gitlab-start[2189]: + mkdir -p /var/lib/secrets gitlab # [ 187.335914] setup-gitlab-start[2189]: + echo nixbot-test-token-of-20-chars gitlab # [ 187.337995] setup-gitlab-start[2189]: + chmod 644 /var/lib/secrets/gitlab-token gitlab # [ 187.340638] setup-gitlab-start[2189]: + curl -fs -X POST -H 'PRIVATE-TOKEN: nixbot-test-token-of-20-chars' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects gitlab # [ 188.143589] gitaly[926]: time="2026-09-04T05:34:56.292Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKCGH0CJTV2A6QZ2VSDQA error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-04T05:35:06.001" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:34:56.292" pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 188.145578] gitaly[926]: time="2026-09-04T05:34:56.294Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKCGH0CJTV2A6QZ2VSDQA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-04T05:35:06.001" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:34:56.292" grpc.time_ms=1.699 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 188.286843] gitaly[926]: time="2026-09-04T05:34:56.436Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKCGH0CJTV2A6QZ2VSDQA error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-04T05:35:06.145" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:34:56.435" pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 188.287710] gitaly[926]: time="2026-09-04T05:34:56.436Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKCGH0CJTV2A6QZ2VSDQA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-04T05:35:06.145" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:34:56.435" grpc.time_ms=0.437 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 188.397067] gitaly[926]: time="2026-09-04T05:34:56.546Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKCGH0CJTV2A6QZ2VSDQA diskcache=ecf4852d-e8c9-4d68-bff0-601114595724 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-04T05:35:06.197" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:34:56.488" pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 188.399214] gitaly[926]: time="2026-09-04T05:34:56.546Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1573 command.oublock=32 command.real_time_ms=23 command.spawn_token_fork_ms=0 command.system_time_ms=17 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKCGH0CJTV2A6QZ2VSDQA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-04T05:35:06.197" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:34:56.488" grpc.time_ms=57.738 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 188.460803] gitaly[926]: time="2026-09-04T05:34:56.609Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKCGH0CJTV2A6QZ2VSDQA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-04T05:35:06.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:34:56.609" pid=926 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 188.462469] gitaly[926]: time="2026-09-04T05:34:56.610Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKCGH0CJTV2A6QZ2VSDQA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-04T05:35:06.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:34:56.609" grpc.time_ms=0.893 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 188.506807] gitaly[926]: time="2026-09-04T05:34:56.656Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKCGH0CJTV2A6QZ2VSDQA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-04T05:35:06.044" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:34:56.644" grpc.time_ms=10.976 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 188.598530] gitaly[926]: time="2026-09-04T05:34:56.747Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKCGH0CJTV2A6QZ2VSDQA error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-04T05:35:06.147" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:34:56.747" pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 188.600302] gitaly[926]: time="2026-09-04T05:34:56.747Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKCGH0CJTV2A6QZ2VSDQA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-04T05:35:06.147" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:34:56.747" grpc.time_ms=0.222 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 188.683385] gitaly[926]: time="2026-09-04T05:34:56.832Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKCGH0CJTV2A6QZ2VSDQA diskcache=7e239d1e-d464-471e-9f38-8f9a109f2381 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-04T05:35:06.188" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:34:56.788" pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 188.685292] gitaly[926]: time="2026-09-04T05:34:56.832Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1585 command.oublock=32 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKCGH0CJTV2A6QZ2VSDQA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-04T05:35:06.188" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:34:56.788" grpc.time_ms=43.834 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 188.719246] gitaly[926]: time="2026-09-04T05:34:56.868Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKCGH0CJTV2A6QZ2VSDQA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-04T05:35:06.267" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:34:56.868" pid=926 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 188.720481] gitaly[926]: time="2026-09-04T05:34:56.868Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKCGH0CJTV2A6QZ2VSDQA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-04T05:35:06.267" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:34:56.868" grpc.time_ms=0.406 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 188.753418] gitaly[926]: time="2026-09-04T05:34:56.902Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKCGH0CJTV2A6QZ2VSDQA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-04T05:35:06.001" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:34:56.892" grpc.time_ms=9.956 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 188.799902] gitaly[926]: time="2026-09-04T05:34:56.949Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKCGH0CJTV2A6QZ2VSDQA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-04T05:35:06.058" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:34:56.948" grpc.time_ms=0.193 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 188.811714] gitaly[926]: time="2026-09-04T05:34:56.960Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKCGH0CJTV2A6QZ2VSDQA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-04T05:35:06.069" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:34:56.960" grpc.time_ms=0.445 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 189.256987] gitaly[926]: time="2026-09-04T05:34:57.405Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKCGH0CJTV2A6QZ2VSDQA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-04T05:35:07.108" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:34:57.399" grpc.time_ms=5.536 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 189.619602] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:34:57 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 2261 gitlab # [ 189.625466] setup-gitlab-start[2189]: + mkdir -p /tmp/test-flake gitlab # [ 189.630156] setup-gitlab-start[2189]: + cd /tmp/test-flake gitlab # [ 189.630935] setup-gitlab-start[2189]: + git init -b master gitlab # [ 189.655113] setup-gitlab-start[2502]: Initialized empty Git repository in /tmp/test-flake/.git/ gitlab # [ 189.657866] setup-gitlab-start[2189]: + git config user.name test gitlab # [ 189.660754] setup-gitlab-start[2189]: + git config user.email test@example.com gitlab # [ 189.666842] setup-gitlab-start[2189]: + cat gitlab # [ 189.671049] setup-gitlab-start[2189]: + git add flake.nix gitlab # [ 189.675763] setup-gitlab-start[2189]: + git commit -m 'initial commit' gitlab # [ 189.693960] setup-gitlab-start[2507]: [master (root-commit) 9e2da04] initial commit gitlab # [ 189.695055] setup-gitlab-start[2507]: 1 file changed, 10 insertions(+) gitlab # [ 189.695743] setup-gitlab-start[2507]: create mode 100644 flake.nix gitlab # [ 189.696907] setup-gitlab-start[2189]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git gitlab # [ 189.703792] setup-gitlab-start[2189]: + git push -u origin master gitlab # [ 189.752454] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:34:57 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 28 gitlab # [ 190.020341] gitaly[926]: time="2026-09-04T05:34:58.169Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKEVKKHSPEBFJYW7CXA3G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:34:58.157" grpc.time_ms=12.087 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 190.022262] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:34:58 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 264 gitlab # [ 190.296066] gitaly[926]: time="2026-09-04T05:34:58.443Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-04T05:35:08.142" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:34:58.432" grpc.time_ms=10.838 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 190.327705] gitaly[926]: time="2026-09-04T05:34:58.476Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=330 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-04T05:35:28.065" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-04T05:34:58.466" grpc.time_ms=10.438 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 190.353081] sidekiq-cluster[2450]: {"severity":"INFO","time":"2026-09-04T05:34:58.485Z","message":"Starting cluster with 1 processes"} gitlab # [ 190.372496] gitaly[926]: time="2026-09-04T05:34:58.521Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 duration_ms=143ns method=POST pid=926 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 190.454044] gitaly[926]: time="2026-09-04T05:34:58.603Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 duration_ms=78ns method=POST pid=926 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 190.460746] gitaly[926]: time="2026-09-04T05:34:58.610Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-04T05:34:58.376" grpc.time_ms=232.874 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 190.508435] gitaly[926]: time="2026-09-04T05:34:58.657Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-04T05:34:58.656" grpc.time_ms=0.778 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 190.556851] gitaly[926]: time="2026-09-04T05:34:58.705Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-04T05:34:58.705" grpc.time_ms=0.588 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 190.588912] gitaly[926]: time="2026-09-04T05:34:58.737Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-04T05:34:58.737" grpc.time_ms=0.239 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 190.681291] gitaly[926]: time="2026-09-04T05:34:58.830Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-04T05:35:08.219" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:34:58.820" grpc.time_ms=10.092 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 190.736908] gitaly[926]: time="2026-09-04T05:34:58.886Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1561 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-04T05:35:08.261" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:34:58.861" grpc.time_ms=24.301 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 190.761391] gitaly[926]: time="2026-09-04T05:34:58.910Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 duration_ms=141ns method=POST pid=926 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 190.766256] gitaly[926]: time="2026-09-04T05:34:58.915Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-04T05:34:58.768" grpc.time_ms=146.731 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 190.787974] gitaly[926]: time="2026-09-04T05:34:58.937Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 diskcache=944402c1-f5e1-4aaa-9612-e015ff54cffa grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:34:58.295" pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 190.788915] gitaly[926]: time="2026-09-04T05:34:58.938Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=209 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13785 command.oublock=32 command.real_time_ms=633 command.spawn_token_fork_ms=0 command.system_time_ms=157 command.user_time_ms=52 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:34:58.295" grpc.time_ms=642.418 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 190.789925] gitlab-workhorse[832]: time="2026-09-04T05:34:58Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 190.790609] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:34:58 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 725 gitlab # [ 190.792256] setup-gitlab-start[2511]: To http://gitlab/root/test-flake.git gitlab # [ 190.792911] setup-gitlab-start[2511]: * [new branch] master -> master gitlab # [ 190.814927] setup-gitlab-start[2511]: branch 'master' set up to track 'origin/master'. gitlab # [ 190.818399] systemd[1]: Finished setup-gitlab.service. gitlab # [ 190.825199] systemd[1]: Starting nixbot CI service... gitlab: (finished: waiting for unit setup-gitlab.service, in 192.36 seconds) gitlab: waiting for unit nixbot.service gitlab # [ 192.006725] nixbot[2595]: {"ts": "2026-09-04T05:35:00.155Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} gitlab # [ 192.034843] nixbot[2595]: {"ts": "2026-09-04T05:35:00.184Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} gitlab # [ 192.061202] nixbot[2595]: {"ts": "2026-09-04T05:35:00.210Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} gitlab # [ 192.066384] nixbot[2595]: {"ts": "2026-09-04T05:35:00.215Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} gitlab # [ 192.071582] nixbot[2595]: {"ts": "2026-09-04T05:35:00.220Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} gitlab # [ 192.079847] nixbot[2595]: {"ts": "2026-09-04T05:35:00.229Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} gitlab # [ 192.084848] nixbot[2595]: {"ts": "2026-09-04T05:35:00.233Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} gitlab # [ 192.090109] nixbot[2595]: {"ts": "2026-09-04T05:35:00.235Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} gitlab # [ 192.098120] nixbot[2595]: {"ts": "2026-09-04T05:35:00.247Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} gitlab # [ 192.105946] nixbot[2595]: {"ts": "2026-09-04T05:35:00.255Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} gitlab # [ 192.118511] nixbot[2595]: {"ts": "2026-09-04T05:35:00.267Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} gitlab # [ 192.131331] nixbot[2595]: {"ts": "2026-09-04T05:35:00.280Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} gitlab # [ 192.136502] nixbot[2595]: {"ts": "2026-09-04T05:35:00.285Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} gitlab # [ 192.144555] nixbot[2595]: {"ts": "2026-09-04T05:35:00.293Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} gitlab # [ 192.150257] nixbot[2595]: {"ts": "2026-09-04T05:35:00.299Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} gitlab # [ 192.158054] nixbot[2595]: {"ts": "2026-09-04T05:35:00.307Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} gitlab # [ 192.161550] nixbot[2595]: {"ts": "2026-09-04T05:35:00.310Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} gitlab # [ 192.170253] nixbot[2595]: {"ts": "2026-09-04T05:35:00.319Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} gitlab # [ 192.179890] nixbot[2595]: {"ts": "2026-09-04T05:35:00.329Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} gitlab # [ 192.186463] nixbot[2595]: {"ts": "2026-09-04T05:35:00.335Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"} gitlab # [ 192.193175] nixbot[2595]: {"ts": "2026-09-04T05:35:00.342Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"} gitlab # [ 192.203217] nixbot[2595]: {"ts": "2026-09-04T05:35:00.352Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"} gitlab # [ 192.209749] nixbot[2595]: {"ts": "2026-09-04T05:35:00.358Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"} gitlab # [ 192.217097] nixbot[2595]: {"ts": "2026-09-04T05:35:00.366Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"} gitlab # [ 192.224371] nixbot[2595]: {"ts": "2026-09-04T05:35:00.373Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"} gitlab # [ 192.228578] nixbot[2595]: {"ts": "2026-09-04T05:35:00.377Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"} gitlab # [ 192.235612] nixbot[2595]: {"ts": "2026-09-04T05:35:00.384Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"} gitlab # [ 192.238475] nixbot[2595]: {"ts": "2026-09-04T05:35:00.387Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"} gitlab # [ 192.245674] nixbot[2595]: {"ts": "2026-09-04T05:35:00.394Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"} gitlab # [ 192.262740] nixbot[2595]: {"ts": "2026-09-04T05:35:00.411Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"} gitlab # [ 192.268930] nixbot[2595]: {"ts": "2026-09-04T05:35:00.418Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"} gitlab # [ 192.275608] nixbot[2595]: {"ts": "2026-09-04T05:35:00.424Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"} gitlab # [ 192.603708] nixbot[2595]: INFO: Started server process [2595] gitlab # [ 192.605397] nixbot[2595]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit) gitlab # [ 192.622564] nixbot[2595]: INFO: ::ffff:127.0.0.1:57234 - "GET /health HTTP/1.1" 200 OK gitlab # [ 192.628691] systemd[1]: Started nixbot CI service. gitlab # [ 192.635331] systemd[1]: Reached target Multi-User System. gitlab # [ 192.638430] systemd[1]: Startup finished in 1.163s (kernel) + 3.702s (initrd) + 3min 7.772s (userspace) = 3min 12.637s. gitlab: (finished: waiting for unit nixbot.service, in 1.04 seconds) gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health gitlab # [ 192.783803] nixbot[2595]: INFO: ::ffff:127.0.0.1:57248 - "GET /health HTTP/1.1" 200 OK gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.04 seconds) subtest: project discovered and webhook registered gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 192.877843] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:01 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 52 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.09 seconds) gitlab # [ 193.271951] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:01 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 646 gitlab # [ 193.274049] nixbot[2595]: {"ts": "2026-09-04T05:35:01.422Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 193.289726] nixbot[2595]: {"ts": "2026-09-04T05:35:01.438Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} gitlab # [ 193.380117] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:01 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 76 gitlab # [ 193.383717] nixbot[2595]: {"ts": "2026-09-04T05:35:01.532Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 193.385915] nixbot[2595]: {"ts": "2026-09-04T05:35:01.535Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} gitlab # [ 193.542189] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:01 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 918 "" "python-httpx/0.28.1" 152 gitlab # [ 193.543584] nixbot[2595]: {"ts": "2026-09-04T05:35:01.692Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""} gitlab # [ 193.688547] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:01 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 135 gitlab # [ 193.690292] nixbot[2595]: {"ts": "2026-09-04T05:35:01.838Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 193.788932] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:01 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "python-httpx/0.28.1" 88 gitlab # [ 193.793499] nixbot[2595]: {"ts": "2026-09-04T05:35:01.942Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 193.863441] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:02 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 918 "" "python-httpx/0.28.1" 64 gitlab # [ 193.868417] nixbot[2595]: {"ts": "2026-09-04T05:35:02.013Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 193.947919] gitaly[926]: time="2026-09-04T05:35:02.096Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKJW2J3CYJYKKB3C4FA20 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-09-04T05:35:32.080" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:02.081" grpc.time_ms=14.367 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.10 seconds) gitlab # [ 193.973663] gitaly[926]: time="2026-09-04T05:35:02.105Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKJW2J3CYJYKKB3C4FA20 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-04T05:35:32.104" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:02.105" grpc.time_ms=0.362 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root (finished: subtest: project discovered and webhook registered, in 1.20 seconds) subtest: push delivers the webhook gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master gitlab # [ 193.985899] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:02 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "curl/8.21.0" 33 gitlab # [ 193.987872] nixbot[2595]: {"ts": "2026-09-04T05:35:02.134Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""} gitlab # [ 193.990284] gitaly[926]: time="2026-09-04T05:35:02.115Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKJW2J3CYJYKKB3C4FA20 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-09-04T05:35:12.008" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:02.108" grpc.time_ms=6.618 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 194.002540] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:02 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 117 gitlab # [ 194.040527] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:02 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 52 gitlab # [ 194.042874] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:02 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 18 gitlab # [ 194.044936] nixbot[2595]: {"ts": "2026-09-04T05:35:02.190Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 194.047469] nixbot[2595]: {"ts": "2026-09-04T05:35:02.194Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} gitlab # [ 194.114849] gitaly[926]: time="2026-09-04T05:35:02.263Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKK2FHFHHCBWHD40W43KF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:02.255" grpc.time_ms=8.17 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 194.116530] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:02 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 41 gitlab # [ 194.206604] gitaly[926]: time="2026-09-04T05:35:02.355Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKK1GVEBD57RJKM48WY3K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:02.345" grpc.time_ms=10.02 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 194.215715] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:02 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 201 "" "git/2.55.0" 172 gitlab # Enumerating objects: 5, done. gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. gitlab # Delta compression using up to 4 threads gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 273.00 KiB/s, done. gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) gitlab # [ 194.256068] gitaly[926]: time="2026-09-04T05:35:02.405Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKK4KB2FDW3GC3K53XNA6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:02.388" pid=926 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc gitlab # [ 194.257903] gitaly[926]: time="2026-09-04T05:35:02.405Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKK4KB2FDW3GC3K53XNA6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:02.388" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 194.258821] gitlab-workhorse[832]: time="2026-09-04T05:35:02Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=113 gitlab # [ 194.261572] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:02 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 115 gitlab # [ 194.261890] gitaly[926]: time="2026-09-04T05:35:02.405Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKK4KB2FDW3GC3K53XNA6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:02.388" grpc.time_ms=16.458 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 194.304083] gitaly[926]: time="2026-09-04T05:35:02.453Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKK88QJE740P88QS6SH4J grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:02.443" pid=926 remote_ip="2001:db8:1::1" request_sha=5edd0d4eb8596d4bf3f68392b35a9e00f1f34abe response_bytes=223 span.kind=server system=grpc gitlab # [ 194.304932] gitaly[926]: time="2026-09-04T05:35:02.453Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1207 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKK88QJE740P88QS6SH4J grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:02.443" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 194.305737] gitaly[926]: time="2026-09-04T05:35:02.453Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1207 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKK88QJE740P88QS6SH4J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:02.443" grpc.time_ms=10.079 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 194.338198] gitlab-workhorse[832]: time="2026-09-04T05:35:02Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=223 gitlab # [ 194.338889] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:02 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 78 gitlab # [ 194.415544] gitaly[926]: time="2026-09-04T05:35:02.564Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKKBYT05BW378TKVP8QQS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:02.557" grpc.time_ms=7.261 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 194.427969] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:02 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 50 gitlab # [ 194.435064] gitaly[926]: time="2026-09-04T05:35:02.583Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=328 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKK81SAWC77RGW3KKMB79 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-04T05:35:32.174" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-04T05:35:02.574" grpc.time_ms=8.772 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 194.458731] gitaly[926]: time="2026-09-04T05:35:02.608Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M1NEKK81SAWC77RGW3KKMB79 duration_ms=73ns method=POST pid=926 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 194.470299] gitaly[926]: time="2026-09-04T05:35:02.619Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKKDNC8B1BG7SMNWP5X8T grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:02.613" pid=926 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 194.472103] gitaly[926]: time="2026-09-04T05:35:02.619Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKKDNC8B1BG7SMNWP5X8T grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:02.613" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 194.472739] gitaly[926]: time="2026-09-04T05:35:02.619Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKKDNC8B1BG7SMNWP5X8T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:02.613" grpc.time_ms=6.06 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 194.486234] gitaly[926]: time="2026-09-04T05:35:02.635Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M1NEKK81SAWC77RGW3KKMB79 duration_ms=24ns method=POST pid=926 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 194.487233] gitaly[926]: time="2026-09-04T05:35:02.635Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1NEKK81SAWC77RGW3KKMB79 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-04T05:35:02.534" grpc.time_ms=101.616 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 194.515559] gitlab-workhorse[832]: time="2026-09-04T05:35:02Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 194.516757] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:02 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 84 gitlab # [ 194.539125] gitaly[926]: time="2026-09-04T05:35:02.688Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1NEKK81SAWC77RGW3KKMB79 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-04T05:35:02.688" grpc.time_ms=0.367 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 194.572225] gitaly[926]: time="2026-09-04T05:35:02.721Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1NEKK81SAWC77RGW3KKMB79 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-04T05:35:02.721" grpc.time_ms=0.312 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 194.593379] gitaly[926]: time="2026-09-04T05:35:02.742Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKKHACT1B89CJ65PBR7DF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:02.732" grpc.time_ms=9.873 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 194.595308] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:02 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 45 gitlab # [ 194.615081] gitaly[926]: time="2026-09-04T05:35:02.763Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1NEKK81SAWC77RGW3KKMB79 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-04T05:35:02.762" grpc.time_ms=0.817 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 194.661085] gitaly[926]: time="2026-09-04T05:35:02.810Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKKJTHBEMRCVA8E8M9SM1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:02.799" pid=926 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc gitlab # [ 194.670564] gitaly[926]: time="2026-09-04T05:35:02.810Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1163 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKKJTHBEMRCVA8E8M9SM1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:02.799" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 194.671551] gitaly[926]: time="2026-09-04T05:35:02.819Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1163 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKKJTHBEMRCVA8E8M9SM1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:02.799" grpc.time_ms=20.31 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 194.672580] gitlab-workhorse[832]: time="2026-09-04T05:35:02Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=209 gitlab # [ 194.673531] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:02 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 75 gitlab # [ 194.706902] gitaly[926]: time="2026-09-04T05:35:02.856Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M1NEKK81SAWC77RGW3KKMB79 duration_ms=65ns method=POST pid=926 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 194.707980] gitaly[926]: time="2026-09-04T05:35:02.857Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1NEKK81SAWC77RGW3KKMB79 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-04T05:35:02.790" grpc.time_ms=67.042 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 194.728785] gitaly[926]: time="2026-09-04T05:35:02.878Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M1NEKK81SAWC77RGW3KKMB79 diskcache=bb52d983-6a36-4168-95c1-2da158fd7851 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:02.478" pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 194.737821] gitaly[926]: time="2026-09-04T05:35:02.887Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=143 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13354 command.oublock=32 command.real_time_ms=389 command.spawn_token_fork_ms=0 command.system_time_ms=94 command.user_time_ms=49 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKK81SAWC77RGW3KKMB79 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:02.478" grpc.time_ms=399.259 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 194.738864] gitlab-workhorse[832]: time="2026-09-04T05:35:02Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 194.739634] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:02 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 486 gitlab # To http://gitlab/root/test-flake.git gitlab # 9e2da04..a1b9203 master -> master gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.78 seconds) gitlab: must succeed: git -C /tmp/test-flake rev-parse master gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 194.811328] gitaly[926]: time="2026-09-04T05:35:02.960Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKKR29D2WR8BPMMZD8SSH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-04T05:35:32.559" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:02.959" grpc.time_ms=0.597 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 194.821466] nixbot[2595]: INFO: ::ffff:127.0.0.1:34832 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.05 seconds) gitlab # [ 194.844068] gitaly[926]: time="2026-09-04T05:35:02.993Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=648 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKKR29D2WR8BPMMZD8SSH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-04T05:35:32.587" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:02.987" grpc.time_ms=5.457 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 194.869252] gitaly[926]: time="2026-09-04T05:35:03.018Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKKR29D2WR8BPMMZD8SSH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-04T05:35:13.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:03.013" grpc.time_ms=5.34 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 194.881585] gitaly[926]: time="2026-09-04T05:35:03.028Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKKR29D2WR8BPMMZD8SSH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-04T05:35:13.013" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:03.024" grpc.time_ms=4.809 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 194.938022] gitaly[926]: time="2026-09-04T05:35:03.087Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKKR29D2WR8BPMMZD8SSH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-04T05:35:13.076" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:35:03.087" grpc.time_ms=0.173 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 194.958946] gitaly[926]: time="2026-09-04T05:35:03.107Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKKR29D2WR8BPMMZD8SSH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-04T05:35:13.092" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:03.102" grpc.time_ms=5.383 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 195.000246] gitaly[926]: time="2026-09-04T05:35:03.149Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1NEKKR29D2WR8BPMMZD8SSH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-04T05:35:03.149" grpc.time_ms=0.305 pid=926 span.kind=server system=grpc gitlab # [ 195.024991] gitaly[926]: time="2026-09-04T05:35:03.174Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1NEKKR29D2WR8BPMMZD8SSH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-04T05:35:03.173" grpc.time_ms=0.351 pid=926 span.kind=server system=grpc gitlab # [ 195.042655] gitaly[926]: time="2026-09-04T05:35:03.191Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKKR29D2WR8BPMMZD8SSH diskcache=ccf661df-fa6a-4cb0-bbd5-b2b0fd3b5373 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-04T05:35:13.111" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:35:03.122" pid=926 ref=refs/keep-around/9e2da041149a9863c5d5c1c3ac743ceb47e9f0d2 remote_ip="2001:db8:1::1" revision=9e2da041149a9863c5d5c1c3ac743ceb47e9f0d2 span.kind=server system=grpc username=root gitlab # [ 195.043620] gitaly[926]: time="2026-09-04T05:35:03.192Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=46 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5315 command.oublock=8 command.real_time_ms=61 command.spawn_token_fork_ms=0 command.system_time_ms=25 command.user_time_ms=21 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKKR29D2WR8BPMMZD8SSH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-04T05:35:13.111" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:35:03.122" grpc.time_ms=69.93 pid=926 ref=refs/keep-around/9e2da041149a9863c5d5c1c3ac743ceb47e9f0d2 remote_ip="2001:db8:1::1" revision=9e2da041149a9863c5d5c1c3ac743ceb47e9f0d2 span.kind=server system=grpc username=root gitlab # [ 195.045890] gitaly[926]: time="2026-09-04T05:35:03.195Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKKR29D2WR8BPMMZD8SSH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-04T05:35:33.094" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:03.194" grpc.time_ms=0.266 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 195.202024] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:03 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/9e2da041149a9863c5d5c1c3ac743ceb47e9f0d2 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 436 gitlab # [ 195.208471] nixbot[2595]: {"ts": "2026-09-04T05:35:03.357Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/9e2da041149a9863c5d5c1c3ac743ceb47e9f0d2 \"HTTP/1.1 201 Created\""} gitlab # [ 195.225559] nixbot[2595]: {"ts": "2026-09-04T05:35:03.374Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-9e2da041149a-0"} gitlab # [ 195.395186] nixbot[2595]: {"ts": "2026-09-04T05:35:03.544Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-9e2da041149a-0", "argv0": "bwrap"} gitlab # [ 195.497601] systemd[1]: Started Nix Daemon. gitlab # [ 195.564395] nix-daemon[2823]: accepted connection from pid 2818, user nixbot gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 195.862552] nixbot[2595]: INFO: ::ffff:127.0.0.1:34836 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 196.901986] nixbot[2595]: INFO: ::ffff:127.0.0.1:34844 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 197.942433] nixbot[2595]: INFO: ::ffff:127.0.0.1:34846 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 198.980438] nixbot[2595]: INFO: ::ffff:127.0.0.1:34860 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 200.017454] nixbot[2595]: INFO: ::ffff:127.0.0.1:34870 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab # [ 200.536739] nix-daemon[2823]: accepted connection from pid 2853, user nixbot gitlab # [ 200.551163] nix-daemon[2823]: accepted connection from pid 2818, user nixbot gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 201.045763] nixbot[2595]: INFO: ::ffff:127.0.0.1:34886 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.03 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 202.073213] nixbot[2595]: INFO: ::ffff:127.0.0.1:34888 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.03 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 203.102203] nixbot[2595]: INFO: ::ffff:127.0.0.1:55966 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.03 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 204.132894] nixbot[2595]: INFO: ::ffff:127.0.0.1:55968 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.03 seconds) gitlab # [ 204.662644] nix-daemon[2823]: accepted connection from pid 2818, user nixbot gitlab # [ 204.745983] nix-daemon[2823]: accepted connection from pid 2882, user nixbot gitlab # [ 204.774778] gitaly[926]: time="2026-09-04T05:35:12.923Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKXFCYR99DPMZBYB81TE5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-04T05:35:42.022" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:12.923" grpc.time_ms=0.545 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 204.867350] gitaly[926]: time="2026-09-04T05:35:13.016Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKXFCYR99DPMZBYB81TE5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-04T05:35:23.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:13.007" grpc.time_ms=8.814 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 204.936459] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:13 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/9e2da041149a9863c5d5c1c3ac743ceb47e9f0d2 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 208 gitlab # [ 204.938024] nixbot[2595]: {"ts": "2026-09-04T05:35:13.086Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/9e2da041149a9863c5d5c1c3ac743ceb47e9f0d2 \"HTTP/1.1 201 Created\""} gitlab # [ 204.980790] gitaly[926]: time="2026-09-04T05:35:13.130Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKXP98YRBMZ1QTYNZT7JM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-04T05:35:43.028" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:13.129" grpc.time_ms=0.732 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.037412] gitaly[926]: time="2026-09-04T05:35:13.186Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKXP98YRBMZ1QTYNZT7JM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-04T05:35:23.166" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:13.177" grpc.time_ms=9.206 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.099692] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:13 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/9e2da041149a9863c5d5c1c3ac743ceb47e9f0d2 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 151 gitlab # [ 205.101300] nixbot[2595]: {"ts": "2026-09-04T05:35:13.249Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/9e2da041149a9863c5d5c1c3ac743ceb47e9f0d2 \"HTTP/1.1 201 Created\""} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.03 seconds) gitlab # [ 205.168830] nixbot[2595]: INFO: ::ffff:127.0.0.1:55984 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 206.200916] nixbot[2595]: INFO: ::ffff:127.0.0.1:55996 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 207.236894] nixbot[2595]: INFO: ::ffff:127.0.0.1:56010 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.03 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 208.273967] nixbot[2595]: INFO: ::ffff:127.0.0.1:56016 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 209.312761] nixbot[2595]: INFO: ::ffff:127.0.0.1:56028 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab # [ 209.765817] nix-daemon[2823]: accepted connection from pid 2925, user nixbot gitlab # [ 209.830393] gitaly[926]: time="2026-09-04T05:35:17.978Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEM2DN2JAC8SQACP5NGNK7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-04T05:35:47.077" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:17.977" grpc.time_ms=0.829 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.885851] gitaly[926]: time="2026-09-04T05:35:18.035Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEM2DN2JAC8SQACP5NGNK7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-04T05:35:28.025" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:18.026" grpc.time_ms=8.893 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.924325] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:18 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/9e2da041149a9863c5d5c1c3ac743ceb47e9f0d2 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 130 gitlab # [ 209.926213] nixbot[2595]: {"ts": "2026-09-04T05:35:18.073Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/9e2da041149a9863c5d5c1c3ac743ceb47e9f0d2 \"HTTP/1.1 201 Created\""} gitlab # [ 210.167564] gitaly[926]: time="2026-09-04T05:35:18.316Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1NEM2QYP67F56RH66AQESMR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:18.308" grpc.time_ms=8.136 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 210.171178] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:18 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 48 gitlab # [ 210.286156] gitaly[926]: time="2026-09-04T05:35:18.434Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEM2SGVDNS2P90302K0E0T grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:18.421" pid=926 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 210.288282] gitaly[926]: time="2026-09-04T05:35:18.435Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEM2SGVDNS2P90302K0E0T grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:18.421" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 210.292773] gitaly[926]: time="2026-09-04T05:35:18.439Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEM2SGVDNS2P90302K0E0T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:18.421" grpc.time_ms=13.387 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 210.294922] gitlab-workhorse[832]: time="2026-09-04T05:35:18Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 210.297220] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:18 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 118 gitlab # [ 210.342955] nixbot[2595]: INFO: ::ffff:127.0.0.1:56040 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab # [ 210.351106] gitaly[926]: time="2026-09-04T05:35:18.500Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEM2XAJBC18AYP1Y7P3S21 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:18.480" pid=926 remote_ip="2001:db8:1::1" request_sha=347aa1ab1b453bb1b9b6d1e870ea78f1069bad98 response_bytes=337 span.kind=server system=grpc gitlab # [ 210.353262] gitaly[926]: time="2026-09-04T05:35:18.500Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1190 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=4 command.system_time_ms=8 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEM2XAJBC18AYP1Y7P3S21 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:18.480" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 210.354075] gitlab-workhorse[832]: time="2026-09-04T05:35:18Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=337 gitlab # [ 210.354808] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:18 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 337 "" "git/2.55.0" 58 gitlab # [ 210.355662] gitaly[926]: time="2026-09-04T05:35:18.500Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1190 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=4 command.system_time_ms=8 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEM2XAJBC18AYP1Y7P3S21 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:18.480" grpc.time_ms=20.736 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.07 seconds) gitlab # [ 210.485681] gitaly[926]: time="2026-09-04T05:35:18.634Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEM31QVN14R2456XGCD6M4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:18.628" grpc.time_ms=6.62 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 210.486679] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:18 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 51 gitlab # [ 210.545679] gitaly[926]: time="2026-09-04T05:35:18.694Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEM33SPHAEEQQZRE8C0430 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:18.684" pid=926 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc gitlab # [ 210.564579] gitaly[926]: time="2026-09-04T05:35:18.695Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEM33SPHAEEQQZRE8C0430 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:18.684" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 210.580495] gitaly[926]: time="2026-09-04T05:35:18.729Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEM33SPHAEEQQZRE8C0430 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:18.684" grpc.time_ms=45.072 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 210.593681] gitlab-workhorse[832]: time="2026-09-04T05:35:18Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 210.594483] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:18 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 93 gitlab # [ 210.680295] gitaly[926]: time="2026-09-04T05:35:18.829Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1NEM37Z8AMN97PA88863YHR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:18.817" grpc.time_ms=12.042 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 210.682536] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:18 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 48 gitlab # [ 210.739027] gitaly[926]: time="2026-09-04T05:35:18.888Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEM39H6VTQ5FG2NYR3VEEJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:18.877" pid=926 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc gitlab # [ 210.758878] gitaly[926]: time="2026-09-04T05:35:18.888Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1175 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEM39H6VTQ5FG2NYR3VEEJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:18.877" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 210.775354] gitaly[926]: time="2026-09-04T05:35:18.924Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1175 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEM39H6VTQ5FG2NYR3VEEJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:18.877" grpc.time_ms=46.767 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 210.788966] gitlab-workhorse[832]: time="2026-09-04T05:35:18Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=216 gitlab # [ 210.789622] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:18 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 104 gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 211.425444] nixbot[2595]: INFO: ::ffff:127.0.0.1:56048 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 212.457541] nixbot[2595]: INFO: ::ffff:127.0.0.1:56052 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.03 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 213.491404] nixbot[2595]: INFO: ::ffff:127.0.0.1:52004 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.03 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.03 seconds) gitlab # [ 214.526222] nixbot[2595]: INFO: ::ffff:127.0.0.1:52012 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab # [ 214.569234] sidekiq-cluster[2545]: 2026-09-04T05:35:22.717Z pid=2545 tid=xuh INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2545, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>446881792} gitlab # [ 214.738304] sidekiq-cluster[2545]: W, [2026-09-04T05:35:22.887403 #2545] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 214.775127] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:22.924Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 215.558202] nixbot[2595]: INFO: ::ffff:127.0.0.1:52028 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab # [ 216.513770] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:24.662Z","message":"GitLab reliable fetch activated!"} gitlab # [ 216.514927] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:24.662Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 216.597772] nixbot[2595]: INFO: ::ffff:127.0.0.1:52032 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 217.636878] nixbot[2595]: INFO: ::ffff:127.0.0.1:52042 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 218.678097] nixbot[2595]: INFO: ::ffff:127.0.0.1:52048 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 219.715670] nixbot[2595]: INFO: ::ffff:127.0.0.1:52064 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 220.752401] nixbot[2595]: INFO: ::ffff:127.0.0.1:52076 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 221.790075] nixbot[2595]: INFO: ::ffff:127.0.0.1:52092 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 222.829805] nixbot[2595]: INFO: ::ffff:127.0.0.1:52098 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 223.869151] nixbot[2595]: INFO: ::ffff:127.0.0.1:39108 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 224.908706] nixbot[2595]: INFO: ::ffff:127.0.0.1:39120 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 225.948256] nixbot[2595]: INFO: ::ffff:127.0.0.1:39130 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 226.988143] nixbot[2595]: INFO: ::ffff:127.0.0.1:39146 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 228.027743] nixbot[2595]: INFO: ::ffff:127.0.0.1:39158 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 229.067594] nixbot[2595]: INFO: ::ffff:127.0.0.1:39168 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 230.108450] nixbot[2595]: INFO: ::ffff:127.0.0.1:39178 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 231.149481] nixbot[2595]: INFO: ::ffff:127.0.0.1:39184 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 232.189379] nixbot[2595]: INFO: ::ffff:127.0.0.1:39194 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 233.229605] nixbot[2595]: INFO: ::ffff:127.0.0.1:40072 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 234.269699] nixbot[2595]: INFO: ::ffff:127.0.0.1:40074 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 235.309360] nixbot[2595]: INFO: ::ffff:127.0.0.1:40076 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab # [ 236.005997] postgres[3109]: [3109] ERROR: relation "table_without_models" does not exist gitlab # [ 236.009140] postgres[3109]: [3109] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 236.349637] nixbot[2595]: INFO: ::ffff:127.0.0.1:40088 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 237.388769] nixbot[2595]: INFO: ::ffff:127.0.0.1:40102 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab # [ 238.331542] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:46.480Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 238.334320] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:46.483Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 238.335946] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:46.485Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 238.338111] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:46.485Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 238.338727] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:46.486Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 238.428297] nixbot[2595]: INFO: ::ffff:127.0.0.1:40118 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 239.466192] nixbot[2595]: INFO: ::ffff:127.0.0.1:40128 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab # [ 239.623668] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:47.771Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"01f68896bc5f878212dd16c5","created_at":"2026-09-04T05:34:56.678Z","trace_propagation_headers":{"sentry-trace":"2cd3717e325a41269e4b94d6f37b4f40-2b9b08cb956b4f45","baggage":"sentry-trace_id=2cd3717e325a41269e4b94d6f37b4f40,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEKCGH0CJTV2A6QZ2VSDQA","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:56.722Z","job_size_bytes":2,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-01f68896bc5f878212dd16c5: start","job_status":"start","queue_duration_s":51.049465,"scheduling_latency_s":51.049535} gitlab # [ 239.769129] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:47.917Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"01f68896bc5f878212dd16c5","created_at":"2026-09-04T05:34:56.678Z","trace_propagation_headers":{"sentry-trace":"2cd3717e325a41269e4b94d6f37b4f40-2b9b08cb956b4f45","baggage":"sentry-trace_id=2cd3717e325a41269e4b94d6f37b4f40,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEKCGH0CJTV2A6QZ2VSDQA","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:56.722Z","job_size_bytes":2,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-01f68896bc5f878212dd16c5: done: 0.146496 sec","job_status":"done","queue_duration_s":51.049465,"scheduling_latency_s":51.049535,"redis_calls":25,"redis_duration_s":0.00582,"redis_read_bytes":912,"redis_write_bytes":2805,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.003237,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001415,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001168,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":11,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.056071,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.081627693,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.146496,"completed_at":"2026-09-04T05:35:47.917Z","load_balancing_strategy":"primary","db_duration_s":0.059588,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 239.772417] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:47.919Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"f931a544cabfb98a5da1d768","created_at":"2026-09-04T05:34:56.735Z","trace_propagation_headers":{"sentry-trace":"2cd3717e325a41269e4b94d6f37b4f40-2b9b08cb956b4f45","baggage":"sentry-trace_id=2cd3717e325a41269e4b94d6f37b4f40,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEKCGH0CJTV2A6QZ2VSDQA","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:56.739Z","job_size_bytes":2,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-f931a544cabfb98a5da1d768: start","job_status":"start","queue_duration_s":51.179935,"scheduling_latency_s":51.179952} gitlab # [ 239.971080] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:48.120Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"f931a544cabfb98a5da1d768","created_at":"2026-09-04T05:34:56.735Z","trace_propagation_headers":{"sentry-trace":"2cd3717e325a41269e4b94d6f37b4f40-2b9b08cb956b4f45","baggage":"sentry-trace_id=2cd3717e325a41269e4b94d6f37b4f40,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEKCGH0CJTV2A6QZ2VSDQA","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:56.739Z","job_size_bytes":2,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-f931a544cabfb98a5da1d768: done: 0.200819 sec","job_status":"done","queue_duration_s":51.179935,"scheduling_latency_s":51.179952,"redis_calls":14,"redis_duration_s":0.004093,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001074,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001736,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001283,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.076977,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.084354855,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.200819,"completed_at":"2026-09-04T05:35:48.119Z","load_balancing_strategy":"primary","db_duration_s":0.02643,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 239.974505] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:48.121Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"fcd3890bc73fc13f59cb07e9","created_at":"2026-09-04T05:34:57.326Z","trace_propagation_headers":{"sentry-trace":"2cd3717e325a41269e4b94d6f37b4f40-2b9b08cb956b4f45","baggage":"sentry-trace_id=2cd3717e325a41269e4b94d6f37b4f40,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEKCGH0CJTV2A6QZ2VSDQA","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:57.331Z","job_size_bytes":5,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-fcd3890bc73fc13f59cb07e9: start","job_status":"start","queue_duration_s":50.790479,"scheduling_latency_s":50.790494} gitlab # [ 240.089291] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:48.238Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"fcd3890bc73fc13f59cb07e9","created_at":"2026-09-04T05:34:57.326Z","trace_propagation_headers":{"sentry-trace":"2cd3717e325a41269e4b94d6f37b4f40-2b9b08cb956b4f45","baggage":"sentry-trace_id=2cd3717e325a41269e4b94d6f37b4f40,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEKCGH0CJTV2A6QZ2VSDQA","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:57.331Z","job_size_bytes":5,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-fcd3890bc73fc13f59cb07e9: done: 0.116563 sec","job_status":"done","queue_duration_s":50.790479,"scheduling_latency_s":50.790494,"redis_calls":12,"redis_duration_s":0.002802,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.0008,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001571,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000431,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.085294,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000288864,"duration_s":0.116563,"completed_at":"2026-09-04T05:35:48.237Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.025044,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 240.091154] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:48.239Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"041dad77751f8bf066de9bc5","created_at":"2026-09-04T05:34:57.351Z","trace_propagation_headers":{"sentry-trace":"2cd3717e325a41269e4b94d6f37b4f40-2b9b08cb956b4f45","baggage":"sentry-trace_id=2cd3717e325a41269e4b94d6f37b4f40,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEKCGH0CJTV2A6QZ2VSDQA","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:57.355Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-041dad77751f8bf066de9bc5: start","job_status":"start","queue_duration_s":50.884243,"scheduling_latency_s":50.884257} gitlab # [ 240.241987] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:48.390Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"041dad77751f8bf066de9bc5","created_at":"2026-09-04T05:34:57.351Z","trace_propagation_headers":{"sentry-trace":"2cd3717e325a41269e4b94d6f37b4f40-2b9b08cb956b4f45","baggage":"sentry-trace_id=2cd3717e325a41269e4b94d6f37b4f40,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEKCGH0CJTV2A6QZ2VSDQA","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:57.355Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-041dad77751f8bf066de9bc5: done: 0.15157 sec","job_status":"done","queue_duration_s":50.884243,"scheduling_latency_s":50.884257,"redis_calls":11,"redis_duration_s":0.0028539999999999998,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000748,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001692,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000414,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.126,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.126,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.081139,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000349765,"duration_s":0.15157,"completed_at":"2026-09-04T05:35:48.390Z","load_balancing_strategy":"primary","db_duration_s":0.070192,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.244649] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:48.392Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"72dff7deeef30369e30ce6ba","created_at":"2026-09-04T05:34:57.357Z","trace_propagation_headers":{"sentry-trace":"2cd3717e325a41269e4b94d6f37b4f40-2b9b08cb956b4f45","baggage":"sentry-trace_id=2cd3717e325a41269e4b94d6f37b4f40,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEKCGH0CJTV2A6QZ2VSDQA","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:57.360Z","job_size_bytes":89,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-72dff7deeef30369e30ce6ba: start","job_status":"start","queue_duration_s":51.031866,"scheduling_latency_s":51.03188} gitlab # [ 240.317081] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:48.418Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"72dff7deeef30369e30ce6ba","created_at":"2026-09-04T05:34:57.357Z","trace_propagation_headers":{"sentry-trace":"2cd3717e325a41269e4b94d6f37b4f40-2b9b08cb956b4f45","baggage":"sentry-trace_id=2cd3717e325a41269e4b94d6f37b4f40,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEKCGH0CJTV2A6QZ2VSDQA","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-04T05:34:57.360Z","job_size_bytes":89,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-72dff7deeef30369e30ce6ba: done: 0.026594 sec","job_status":"done","queue_duration_s":51.031866,"scheduling_latency_s":51.03188,"redis_calls":11,"redis_duration_s":0.00245,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000705,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001286,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000459,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01514,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000291936,"duration_s":0.026594,"completed_at":"2026-09-04T05:35:48.418Z","load_balancing_strategy":"primary","db_duration_s":0.005793,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.330674] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:48.420Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 9e2da041149a9863c5d5c1c3ac743ceb47e9f0d2 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"97c2f8433b353dba866edcb3","created_at":"2026-09-04T05:34:58.805Z","trace_propagation_headers":{"sentry-trace":"bc8395dfe7614f3283547c5e57875672-72cdc280ecbe4fe2","baggage":"sentry-trace_id=bc8395dfe7614f3283547c5e57875672,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEKF552Z7PTSMVKD65DJA5","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:58.808Z","job_size_bytes":185,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-97c2f8433b353dba866edcb3: start","job_status":"start","queue_duration_s":49.611443,"scheduling_latency_s":49.611457} gitlab # [ 240.347107] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:48.431Z","message":"changes: \"0000000000000000000000000000000000000000 9e2da041149a9863c5d5c1c3ac743ceb47e9f0d2 refs/heads/master\\n\""} gitlab # [ 240.349852] gitaly[926]: time="2026-09-04T05:35:48.461Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-04T05:35:58.000" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:48.451" grpc.time_ms=9.788 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 240.400723] gitaly[926]: time="2026-09-04T05:35:48.474Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-04T05:35:58.014" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:35:48.465" grpc.time_ms=9.603 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 240.401607] gitaly[926]: time="2026-09-04T05:35:48.521Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=4 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-04T05:35:58.057" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:48.508" grpc.time_ms=13.217 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 240.408748] gitaly[926]: time="2026-09-04T05:35:48.558Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3933 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-04T05:35:48.557" grpc.time_ms=0.275 pid=926 span.kind=server system=grpc gitlab # [ 240.432310] gitaly[926]: time="2026-09-04T05:35:48.581Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3935 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-04T05:35:48.581" grpc.time_ms=0.31 pid=926 span.kind=server system=grpc gitlab # [ 240.442410] gitaly[926]: time="2026-09-04T05:35:48.591Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 diskcache=821520b7-e33c-4258-ab73-9cc894cdef1f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-04T05:35:58.077" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:35:48.528" pid=926 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 240.443390] gitaly[926]: time="2026-09-04T05:35:48.591Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=64 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5229 command.oublock=8 command.real_time_ms=57 command.spawn_token_fork_ms=0 command.system_time_ms=40 command.user_time_ms=24 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-04T05:35:58.077" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:35:48.528" grpc.time_ms=63.638 pid=926 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 240.453721] gitaly[926]: time="2026-09-04T05:35:48.602Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-04T05:35:58.152" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:35:48.602" grpc.time_ms=0.196 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 240.463851] gitaly[926]: time="2026-09-04T05:35:48.613Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=944 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-04T05:35:58.155" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:48.605" grpc.time_ms=7.698 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 240.505514] nixbot[2595]: INFO: ::ffff:127.0.0.1:40130 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab # [ 240.525779] gitaly[926]: time="2026-09-04T05:35:48.675Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=337 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-09-04T05:36:18.069" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:48.669" grpc.time_ms=5.188 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 240.548318] gitaly[926]: time="2026-09-04T05:35:48.697Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-04T05:35:58.246" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:35:48.697" grpc.time_ms=0.256 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 240.559154] gitaly[926]: time="2026-09-04T05:35:48.703Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=345 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-04T05:35:58.248" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-09-04T05:35:48.699" grpc.time_ms=4.2 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 240.575474] gitaly[926]: time="2026-09-04T05:35:48.724Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-04T05:35:58.267" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:48.718" grpc.time_ms=6.624 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 240.596462] gitaly[926]: time="2026-09-04T05:35:48.745Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-04T05:35:58.289" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:48.739" grpc.time_ms=5.875 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 240.627976] gitaly[926]: time="2026-09-04T05:35:48.777Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=968 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-04T05:36:18.166" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:48.767" grpc.time_ms=9.898 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 240.976899] gitaly[926]: time="2026-09-04T05:35:49.126Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-04T05:35:59.003" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:49.124" grpc.time_ms=1.568 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 241.055126] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:49.203Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 9e2da041149a9863c5d5c1c3ac743ceb47e9f0d2 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"97c2f8433b353dba866edcb3","created_at":"2026-09-04T05:34:58.805Z","trace_propagation_headers":{"sentry-trace":"bc8395dfe7614f3283547c5e57875672-72cdc280ecbe4fe2","baggage":"sentry-trace_id=bc8395dfe7614f3283547c5e57875672,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEKF552Z7PTSMVKD65DJA5","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:34:58.808Z","job_size_bytes":185,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-97c2f8433b353dba866edcb3: done: 0.782932 sec","job_status":"done","queue_duration_s":49.611443,"scheduling_latency_s":49.611457,"gitaly_calls":13,"gitaly_duration_s":0.206803,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.064148,"redis_read_bytes":2103,"redis_write_bytes":21750,"redis_cache_calls":15,"redis_cache_duration_s":0.003101,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.006409,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001376,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7838,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.004644,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.035903,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.012715,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.049,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.173,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.193,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.049,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.445769,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00044447,"exclusive_lock_hold_duration_s":0.023815595999999998,"duration_s":0.782932,"completed_at":"2026-09-04T05:35:49.202Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.086836,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 241.069646] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:49.205Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","9e2da041149a9863c5d5c1c3ac743ceb47e9f0d2 a1b920324eb15e51aadde4b427f3f24773bfca94 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"b16dd5e3acd1437b6d6fa553","created_at":"2026-09-04T05:35:02.824Z","trace_propagation_headers":{"sentry-trace":"06f1c1278e884e529d8ea0deb2f12caf-bd21d8dab983470a","baggage":"sentry-trace_id=06f1c1278e884e529d8ea0deb2f12caf,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEKK81SAWC77RGW3KKMB79","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:35:02.827Z","job_size_bytes":185,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-b16dd5e3acd1437b6d6fa553: start","job_status":"start","queue_duration_s":46.378052,"scheduling_latency_s":46.378088} gitlab # [ 241.135694] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:49.220Z","message":"changes: \"9e2da041149a9863c5d5c1c3ac743ceb47e9f0d2 a1b920324eb15e51aadde4b427f3f24773bfca94 refs/heads/master\\n\""} gitlab # [ 241.136522] gitaly[926]: time="2026-09-04T05:35:49.257Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKK81SAWC77RGW3KKMB79 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-04T05:35:59.127" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:49.248" grpc.time_ms=8.986 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 241.179903] gitaly[926]: time="2026-09-04T05:35:49.329Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKK81SAWC77RGW3KKMB79 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-04T05:35:59.197" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:35:49.317" grpc.time_ms=11.223 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 241.186968] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:49.335Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"391c5753e30b60584aae3f42","created_at":"2026-09-04T05:35:49.332Z","trace_propagation_headers":{"sentry-trace":"06f1c1278e884e529d8ea0deb2f12caf-6b1f5a68b644459e","baggage":"sentry-trace_id=06f1c1278e884e529d8ea0deb2f12caf,sentry-release=Unknown"},"correlation_id":"01M1NEKK81SAWC77RGW3KKMB79","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"47b768a5e6bd0789c91d202a","job_size_bytes":46,"pid":2545,"job_status":"deduplicated","message":"ProjectCacheWorker JID-391c5753e30b60584aae3f42: deduplicated: until executing","deduplication.type":"until executing"} gitlab # [ 241.218513] gitaly[926]: time="2026-09-04T05:35:49.367Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKK81SAWC77RGW3KKMB79 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-04T05:35:59.235" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:49.356" grpc.time_ms=11.041 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 241.242875] gitaly[926]: time="2026-09-04T05:35:49.392Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=657 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKK81SAWC77RGW3KKMB79 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-09-04T05:36:19.080" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:49.380" grpc.time_ms=10.936 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 241.264308] gitaly[926]: time="2026-09-04T05:35:49.413Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=348 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKK81SAWC77RGW3KKMB79 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-04T05:35:59.286" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-09-04T05:35:49.406" grpc.time_ms=6.47 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 241.296871] gitaly[926]: time="2026-09-04T05:35:49.446Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKK81SAWC77RGW3KKMB79 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-04T05:35:59.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:49.436" grpc.time_ms=9.159 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 241.466661] gitaly[926]: time="2026-09-04T05:35:49.615Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKK81SAWC77RGW3KKMB79 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-04T05:35:59.105" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:49.556" grpc.time_ms=58.878 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.03 seconds) gitlab # [ 241.486609] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:49.635Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","9e2da041149a9863c5d5c1c3ac743ceb47e9f0d2 a1b920324eb15e51aadde4b427f3f24773bfca94 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"b16dd5e3acd1437b6d6fa553","created_at":"2026-09-04T05:35:02.824Z","trace_propagation_headers":{"sentry-trace":"06f1c1278e884e529d8ea0deb2f12caf-bd21d8dab983470a","baggage":"sentry-trace_id=06f1c1278e884e529d8ea0deb2f12caf,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEKK81SAWC77RGW3KKMB79","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-04T05:35:02.827Z","job_size_bytes":185,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-b16dd5e3acd1437b6d6fa553: done: 0.429489 sec","job_status":"done","queue_duration_s":46.378052,"scheduling_latency_s":46.378088,"gitaly_calls":7,"gitaly_duration_s":0.217736,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.028202,"redis_read_bytes":500,"redis_write_bytes":11941,"redis_cache_calls":9,"redis_cache_duration_s":0.003132,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.001243,"redis_queues_read_bytes":8,"redis_queues_write_bytes":5718,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.004529,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.009612,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.009686,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.03,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.03,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.225401,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000983365,"exclusive_lock_hold_duration_s":0.076021114,"duration_s":0.429489,"completed_at":"2026-09-04T05:35:49.635Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.030788,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 241.550341] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:49.637Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"43df6f3ff840aeef3cc138df","created_at":"2026-09-04T05:35:03.229Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-d2d75fdb3b3d45c4","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:03.233Z","job_size_bytes":65,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-43df6f3ff840aeef3cc138df: start","job_status":"start","queue_duration_s":46.403886,"scheduling_latency_s":46.40393} gitlab # [ 241.551459] nixbot[2595]: INFO: ::ffff:127.0.0.1:40132 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab # [ 241.580537] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:49.729Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"43df6f3ff840aeef3cc138df","created_at":"2026-09-04T05:35:03.229Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-d2d75fdb3b3d45c4","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:03.233Z","job_size_bytes":65,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-43df6f3ff840aeef3cc138df: done: 0.091934 sec","job_status":"done","queue_duration_s":46.403886,"scheduling_latency_s":46.40393,"redis_calls":16,"redis_duration_s":0.004672,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002199,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001914,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000559,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.058019,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000339708,"duration_s":0.091934,"completed_at":"2026-09-04T05:35:49.729Z","load_balancing_strategy":"primary","db_duration_s":0.024166,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 241.581989] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:49.731Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"59dd1228770506737a208f81","created_at":"2026-09-04T05:35:03.234Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-d2d75fdb3b3d45c4","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:03.236Z","job_size_bytes":65,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-59dd1228770506737a208f81: start","job_status":"start","queue_duration_s":46.494367,"scheduling_latency_s":46.494382} gitlab # [ 241.661485] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:49.810Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"59dd1228770506737a208f81","created_at":"2026-09-04T05:35:03.234Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-d2d75fdb3b3d45c4","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:03.236Z","job_size_bytes":65,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-59dd1228770506737a208f81: done: 0.078786 sec","job_status":"done","queue_duration_s":46.494367,"scheduling_latency_s":46.494382,"redis_calls":21,"redis_duration_s":0.004887000000000001,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.002001,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001526,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00136,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042809,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000361219,"duration_s":0.078786,"completed_at":"2026-09-04T05:35:49.809Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.025262,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.663827] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:49.812Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"48ae5be9cfe704a1ddc204ef","created_at":"2026-09-04T05:35:03.334Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-d2d75fdb3b3d45c4","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:03.338Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-48ae5be9cfe704a1ddc204ef: start","job_status":"start","queue_duration_s":46.473607,"scheduling_latency_s":46.473646} gitlab # [ 241.839311] gitaly[926]: time="2026-09-04T05:35:49.988Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1NEKKR29D2WR8BPMMZD8SSH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-04T05:35:49.987" grpc.time_ms=1.105 pid=926 span.kind=server system=grpc gitlab # [ 241.896186] gitaly[926]: time="2026-09-04T05:35:50.044Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1NEKKR29D2WR8BPMMZD8SSH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-04T05:35:50.044" grpc.time_ms=0.538 pid=926 span.kind=server system=grpc gitlab # [ 241.941704] gitaly[926]: time="2026-09-04T05:35:50.090Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1NEKKR29D2WR8BPMMZD8SSH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-04T05:35:50.090" grpc.time_ms=0.245 pid=926 span.kind=server system=grpc gitlab # [ 241.968036] gitaly[926]: time="2026-09-04T05:35:50.117Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKKR29D2WR8BPMMZD8SSH diskcache=8ea6b6a4-886a-4b8d-b198-10b359994411 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-04T05:36:19.344" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:49.945" pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 241.977623] gitaly[926]: time="2026-09-04T05:35:50.126Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=143 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=6999 command.oublock=0 command.real_time_ms=164 command.spawn_token_fork_ms=0 command.system_time_ms=91 command.user_time_ms=52 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKKR29D2WR8BPMMZD8SSH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-04T05:36:19.344" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:49.945" grpc.time_ms=172.212 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 242.092956] gitaly[926]: time="2026-09-04T05:35:50.242Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKKR29D2WR8BPMMZD8SSH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-04T05:36:20.040" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:50.240" grpc.time_ms=1.24 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 242.161812] gitaly[926]: time="2026-09-04T05:35:50.310Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKKR29D2WR8BPMMZD8SSH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-04T05:36:20.109" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:50.309" grpc.time_ms=1.096 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 242.177726] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:50.326Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"48ae5be9cfe704a1ddc204ef","created_at":"2026-09-04T05:35:03.334Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-d2d75fdb3b3d45c4","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:03.338Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2545,"job_status":"rescheduled","message":"PipelineProcessWorker JID-48ae5be9cfe704a1ddc204ef: rescheduled"} gitlab # [ 242.188466] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:50.337Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"48ae5be9cfe704a1ddc204ef","created_at":"2026-09-04T05:35:03.334Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-d2d75fdb3b3d45c4","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:03.338Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-48ae5be9cfe704a1ddc204ef: done: 0.524809 sec","job_status":"done","queue_duration_s":46.473607,"scheduling_latency_s":46.473646,"gitaly_calls":3,"gitaly_duration_s":0.194945,"redis_calls":120,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.031234,"redis_read_bytes":462,"redis_write_bytes":29288,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001109,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001951,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.011427,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":27,"redis_queues_duration_s":0.004571,"redis_queues_read_bytes":39,"redis_queues_write_bytes":15575,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.006684,"redis_queues_metadata_read_bytes":34,"redis_queues_metadata_write_bytes":3577,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000527,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000783,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003073,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.078,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.018,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.021,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.047,"db_ci_duration_s":0.03,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.219778,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.45270893500000003,"duration_s":0.524809,"completed_at":"2026-09-04T05:35:50.337Z","load_balancing_strategy":"primary","db_duration_s":0.089625,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 242.249765] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:50.339Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"47b768a5e6bd0789c91d202a","created_at":"2026-09-04T05:35:48.477Z","trace_propagation_headers":{"sentry-trace":"bc8395dfe7614f3283547c5e57875672-9caefd82d47141dc","baggage":"sentry-trace_id=bc8395dfe7614f3283547c5e57875672,sentry-release=Unknown"},"correlation_id":"01M1NEKF552Z7PTSMVKD65DJA5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:48.489Z","job_size_bytes":46,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-47b768a5e6bd0789c91d202a: start","job_status":"start","queue_duration_s":1.849447,"scheduling_latency_s":1.849474} gitlab # [ 242.250644] gitaly[926]: time="2026-09-04T05:35:50.376Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-09-04T11:35:50.370" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:35:50.371" grpc.time_ms=4.974 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 242.251544] gitaly[926]: time="2026-09-04T05:35:50.395Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=652 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-04T05:36:20.188" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:50.389" grpc.time_ms=6.137 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 242.280971] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:50.429Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"47b768a5e6bd0789c91d202a","created_at":"2026-09-04T05:35:48.477Z","trace_propagation_headers":{"sentry-trace":"bc8395dfe7614f3283547c5e57875672-9caefd82d47141dc","baggage":"sentry-trace_id=bc8395dfe7614f3283547c5e57875672,sentry-release=Unknown"},"correlation_id":"01M1NEKF552Z7PTSMVKD65DJA5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:48.489Z","job_size_bytes":46,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-47b768a5e6bd0789c91d202a: done: 0.090666 sec","job_status":"done","queue_duration_s":1.849447,"scheduling_latency_s":1.849474,"gitaly_calls":2,"gitaly_duration_s":0.016297,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.008264,"redis_read_bytes":69,"redis_write_bytes":4879,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001901,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000742,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2112,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001742,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.003111,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":748,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000768,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.051869,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034334,"duration_s":0.090666,"completed_at":"2026-09-04T05:35:50.429Z","load_balancing_strategy":"primary","db_duration_s":0.013334,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 242.334414] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:50.432Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"3e47ca57e72cb6a95ef76832","created_at":"2026-09-04T05:35:48.501Z","trace_propagation_headers":{"sentry-trace":"bc8395dfe7614f3283547c5e57875672-9caefd82d47141dc","baggage":"sentry-trace_id=bc8395dfe7614f3283547c5e57875672,sentry-release=Unknown"},"correlation_id":"01M1NEKF552Z7PTSMVKD65DJA5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:48.504Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-3e47ca57e72cb6a95ef76832: start","job_status":"start","queue_duration_s":1.92715,"scheduling_latency_s":1.927176} gitlab # [ 242.374694] gitaly[926]: time="2026-09-04T05:35:50.523Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 diskcache=7bbdb69e-de0e-4a1c-9811-1c8ab0b2b428 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-04T11:35:50.451" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:50.451" pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 242.375876] gitaly[926]: time="2026-09-04T05:35:50.524Z" level=info msg="finished unary call with code OK" catfile.duration_ms=48 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=48 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1283 command.oublock=0 command.real_time_ms=70 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-04T11:35:50.451" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:50.451" grpc.time_ms=72.246 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 242.414377] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:50.563Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"3e47ca57e72cb6a95ef76832","created_at":"2026-09-04T05:35:48.501Z","trace_propagation_headers":{"sentry-trace":"bc8395dfe7614f3283547c5e57875672-9caefd82d47141dc","baggage":"sentry-trace_id=bc8395dfe7614f3283547c5e57875672,sentry-release=Unknown"},"correlation_id":"01M1NEKF552Z7PTSMVKD65DJA5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:48.504Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-3e47ca57e72cb6a95ef76832: done: 0.131203 sec","job_status":"done","queue_duration_s":1.92715,"scheduling_latency_s":1.927176,"gitaly_calls":1,"gitaly_duration_s":0.074376,"redis_calls":15,"redis_duration_s":0.003921,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000419,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001503,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000625,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001374,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.015,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.023,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032545,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.110051139,"duration_s":0.131203,"completed_at":"2026-09-04T05:35:50.563Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018705,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.455540] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:50.564Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"9f29fcb334b8ebd8f52b9b25","created_at":"2026-09-04T05:35:48.599Z","trace_propagation_headers":{"sentry-trace":"bc8395dfe7614f3283547c5e57875672-9caefd82d47141dc","baggage":"sentry-trace_id=bc8395dfe7614f3283547c5e57875672,sentry-release=Unknown"},"correlation_id":"01M1NEKF552Z7PTSMVKD65DJA5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:48.600Z","job_size_bytes":23,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-9f29fcb334b8ebd8f52b9b25: start","job_status":"start","queue_duration_s":1.964069,"scheduling_latency_s":1.964091} gitlab # [ 242.483483] gitaly[926]: time="2026-09-04T05:35:50.631Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=648 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-04T05:36:20.420" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:50.620" grpc.time_ms=10.734 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 242.494651] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:50.643Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"9f29fcb334b8ebd8f52b9b25","created_at":"2026-09-04T05:35:48.599Z","trace_propagation_headers":{"sentry-trace":"bc8395dfe7614f3283547c5e57875672-9caefd82d47141dc","baggage":"sentry-trace_id=bc8395dfe7614f3283547c5e57875672,sentry-release=Unknown"},"correlation_id":"01M1NEKF552Z7PTSMVKD65DJA5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:48.600Z","job_size_bytes":23,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-9f29fcb334b8ebd8f52b9b25: done: 0.078702 sec","job_status":"done","queue_duration_s":1.964069,"scheduling_latency_s":1.964091,"gitaly_calls":1,"gitaly_duration_s":0.049781,"redis_calls":16,"redis_duration_s":0.004353,"redis_read_bytes":52,"redis_write_bytes":2048,"redis_queues_calls":1,"redis_queues_duration_s":0.000394,"redis_queues_read_bytes":1,"redis_queues_write_bytes":979,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001214,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001956,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000789,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020837,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.078702,"completed_at":"2026-09-04T05:35:50.643Z","load_balancing_strategy":"primary","db_duration_s":0.002888,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 242.496676] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:50.645Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"6d6d3f99ae5f7637bf3cf2d8","created_at":"2026-09-04T05:35:49.007Z","trace_propagation_headers":{"sentry-trace":"bc8395dfe7614f3283547c5e57875672-9caefd82d47141dc","baggage":"sentry-trace_id=bc8395dfe7614f3283547c5e57875672,sentry-release=Unknown"},"correlation_id":"01M1NEKF552Z7PTSMVKD65DJA5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:91cc3d1cecf116b0866238f55d114f7b036dcdd1ee015f23f268e362ed444f2f","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:49.013Z","job_size_bytes":467,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-6d6d3f99ae5f7637bf3cf2d8: start","job_status":"start","queue_duration_s":1.632103,"scheduling_latency_s":1.632186} gitlab # [ 242.554062] gitaly[926]: time="2026-09-04T05:35:50.703Z" level=info msg="finished unary call with code OK" catfile.duration_ms=35 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=35 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-04T05:36:20.465" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:50.666" grpc.time_ms=36.413 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 242.569581] gitaly[926]: time="2026-09-04T05:35:50.718Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-04T05:36:00.276" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:50.707" grpc.time_ms=11.187 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 242.581346] nixbot[2595]: INFO: ::ffff:127.0.0.1:40140 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab # [ 242.664585] gitaly[926]: time="2026-09-04T05:35:50.813Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-04T05:36:20.612" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:50.812" grpc.time_ms=0.782 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 242.697866] gitaly[926]: time="2026-09-04T05:35:50.847Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=42 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-04T05:36:20.643" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:50.843" grpc.time_ms=3.02 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 242.796825] gitaly[926]: time="2026-09-04T05:35:50.946Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-04T05:36:00.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:50.936" grpc.time_ms=9.133 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 243.183412] gitaly[926]: time="2026-09-04T05:35:51.332Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=966 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-04T05:36:21.113" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:51.314" grpc.time_ms=18.256 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 243.236703] gitaly[926]: time="2026-09-04T05:35:51.385Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKF552Z7PTSMVKD65DJA5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-04T05:36:21.170" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:51.371" grpc.time_ms=14.467 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 243.299699] sidekiq-cluster[2545]: {"severity":"WARN","time":"2026-09-04T05:35:51.447Z","class":"CreatePipelineWorker","correlation_id":"01M1NEKF552Z7PTSMVKD65DJA5","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"9e2da041149a9863c5d5c1c3ac743ceb47e9f0d2","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"9e2da041149a9863c5d5c1c3ac743ceb47e9f0d2","ref":"refs/heads/master"},"retry":0} gitlab # [ 243.309209] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:51.457Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"6d6d3f99ae5f7637bf3cf2d8","created_at":"2026-09-04T05:35:49.007Z","trace_propagation_headers":{"sentry-trace":"bc8395dfe7614f3283547c5e57875672-9caefd82d47141dc","baggage":"sentry-trace_id=bc8395dfe7614f3283547c5e57875672,sentry-release=Unknown"},"correlation_id":"01M1NEKF552Z7PTSMVKD65DJA5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:91cc3d1cecf116b0866238f55d114f7b036dcdd1ee015f23f268e362ed444f2f","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:49.013Z","job_size_bytes":467,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-6d6d3f99ae5f7637bf3cf2d8: done: 0.812491 sec","job_status":"done","queue_duration_s":1.632103,"scheduling_latency_s":1.632186,"gitaly_calls":7,"gitaly_duration_s":0.21084,"redis_calls":51,"redis_duration_s":0.039198,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.000983,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.004708,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001611,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.030503,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001393,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.508434,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004928,"duration_s":0.812491,"completed_at":"2026-09-04T05:35:51.457Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.053584,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 243.370972] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:51.460Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"03239c04-ba7e-4f83-bec3-2920d1b88e66\"}"],"class":"WebHookWorker","jid":"c880ae85d793e163729fad81","created_at":"2026-09-04T05:35:49.100Z","trace_propagation_headers":{"sentry-trace":"bc8395dfe7614f3283547c5e57875672-9caefd82d47141dc","baggage":"sentry-trace_id=bc8395dfe7614f3283547c5e57875672,sentry-release=Unknown"},"correlation_id":"01M1NEKF552Z7PTSMVKD65DJA5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:beb7015d75c1ce583439f36c6048b7b1c9054f36ed45cb4cfb8b3e9f07a9a164","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:49.105Z","job_size_bytes":1751,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-c880ae85d793e163729fad81: start","job_status":"start","queue_duration_s":2.355009,"scheduling_latency_s":2.355047} gitlab # [ 243.422883] nixbot[2595]: INFO: ::ffff:127.0.0.1:57796 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 243.442189] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:51.590Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"03239c04-ba7e-4f83-bec3-2920d1b88e66\"}"],"class":"WebHookWorker","jid":"c880ae85d793e163729fad81","created_at":"2026-09-04T05:35:49.100Z","trace_propagation_headers":{"sentry-trace":"bc8395dfe7614f3283547c5e57875672-9caefd82d47141dc","baggage":"sentry-trace_id=bc8395dfe7614f3283547c5e57875672,sentry-release=Unknown"},"correlation_id":"01M1NEKF552Z7PTSMVKD65DJA5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:beb7015d75c1ce583439f36c6048b7b1c9054f36ed45cb4cfb8b3e9f07a9a164","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:49.105Z","job_size_bytes":1751,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-c880ae85d793e163729fad81: done: 0.130611 sec","job_status":"done","queue_duration_s":2.355009,"scheduling_latency_s":2.355047,"redis_calls":19,"redis_duration_s":0.005124,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001306,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000343,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001983,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001492,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.04912998799997581,"cpu_s":0.067103,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000476876,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.130611,"completed_at":"2026-09-04T05:35:51.590Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003338,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.490803] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:51.639Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"603870faeccb058680fc3840","created_at":"2026-09-04T05:35:49.343Z","trace_propagation_headers":{"sentry-trace":"06f1c1278e884e529d8ea0deb2f12caf-6b1f5a68b644459e","baggage":"sentry-trace_id=06f1c1278e884e529d8ea0deb2f12caf,sentry-release=Unknown"},"correlation_id":"01M1NEKK81SAWC77RGW3KKMB79","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"3e47ca57e72cb6a95ef76832","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:49.346Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-603870faeccb058680fc3840: start","job_status":"start","queue_duration_s":2.292821,"scheduling_latency_s":2.292869} gitlab # [ 243.527983] gitaly[926]: time="2026-09-04T05:35:51.676Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEN39MJBJ4MGBXS4CN0R4F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:51.667" grpc.time_ms=9.037 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 243.529867] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:51 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 72 gitlab # [ 243.531522] gitaly[926]: time="2026-09-04T05:35:51.680Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKK81SAWC77RGW3KKMB79 diskcache=0ed21f0e-7268-493a-8e82-31ac88f37cd6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-04T11:35:51.666" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:51.667" pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 243.532978] gitaly[926]: time="2026-09-04T05:35:51.681Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=313 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKK81SAWC77RGW3KKMB79 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-04T11:35:51.666" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:51.667" grpc.time_ms=13.255 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 243.551466] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:51.700Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"603870faeccb058680fc3840","created_at":"2026-09-04T05:35:49.343Z","trace_propagation_headers":{"sentry-trace":"06f1c1278e884e529d8ea0deb2f12caf-6b1f5a68b644459e","baggage":"sentry-trace_id=06f1c1278e884e529d8ea0deb2f12caf,sentry-release=Unknown"},"correlation_id":"01M1NEKK81SAWC77RGW3KKMB79","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"3e47ca57e72cb6a95ef76832","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:49.346Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-603870faeccb058680fc3840: done: 0.060757 sec","job_status":"done","queue_duration_s":2.292821,"scheduling_latency_s":2.292869,"gitaly_calls":1,"gitaly_duration_s":0.016961,"redis_calls":12,"redis_duration_s":0.0030759999999999997,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001473,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000596,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001007,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032227,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.034398506,"duration_s":0.060757,"completed_at":"2026-09-04T05:35:51.700Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005968,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab # [ 243.598700] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:51.701Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"855436918c2a6099ef166773","created_at":"2026-09-04T05:35:49.513Z","trace_propagation_headers":{"sentry-trace":"06f1c1278e884e529d8ea0deb2f12caf-6b1f5a68b644459e","baggage":"sentry-trace_id=06f1c1278e884e529d8ea0deb2f12caf,sentry-release=Unknown"},"correlation_id":"01M1NEKK81SAWC77RGW3KKMB79","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cafd6615b3319cfcb8412e780b08bb275d66861d10b134d021519aa235541a7e","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:49.516Z","job_size_bytes":467,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-855436918c2a6099ef166773: start","job_status":"start","queue_duration_s":2.185138,"scheduling_latency_s":2.185164} gitlab # [ 243.599547] gitaly[926]: time="2026-09-04T05:35:51.720Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKK81SAWC77RGW3KKMB79 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-04T05:36:21.518" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:51.719" grpc.time_ms=0.6 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 243.600304] gitaly[926]: time="2026-09-04T05:35:51.727Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKK81SAWC77RGW3KKMB79 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-04T05:36:01.001" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:51.721" grpc.time_ms=5.196 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 243.606359] nixbot[2595]: INFO: ::ffff:127.0.0.1:57812 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab # [ 243.613997] gitaly[926]: time="2026-09-04T05:35:51.763Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKK81SAWC77RGW3KKMB79 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-04T05:36:21.562" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:51.762" grpc.time_ms=0.502 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 243.620049] gitaly[926]: time="2026-09-04T05:35:51.769Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=287 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKK81SAWC77RGW3KKMB79 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-04T05:36:21.565" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:51.766" grpc.time_ms=2.937 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 243.661191] gitaly[926]: time="2026-09-04T05:35:51.794Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKK81SAWC77RGW3KKMB79 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-04T05:36:01.066" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:51.786" grpc.time_ms=7.08 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 243.735138] gitaly[926]: time="2026-09-04T05:35:51.884Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN3GX6SXDRBGG0XCG8YX9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:51.876" pid=926 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 243.735958] gitaly[926]: time="2026-09-04T05:35:51.884Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN3GX6SXDRBGG0XCG8YX9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:51.876" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 243.736679] gitaly[926]: time="2026-09-04T05:35:51.884Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN3GX6SXDRBGG0XCG8YX9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:51.876" grpc.time_ms=8.198 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 243.737512] gitlab-workhorse[832]: time="2026-09-04T05:35:51Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 243.737968] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:51 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 47 gitlab # [ 243.792926] nixbot[2595]: {"ts": "2026-09-04T05:35:51.942Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"} gitlab # [ 243.845486] gitaly[926]: time="2026-09-04T05:35:51.994Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN3MKVVQEMZXMWN2P74EH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-04T05:36:21.093" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:51.993" grpc.time_ms=0.822 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 243.908588] gitaly[926]: time="2026-09-04T05:35:52.057Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEN3MKVVQEMZXMWN2P74EH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-04T05:36:02.049" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:52.049" grpc.time_ms=7.949 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 243.975703] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:52 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/9e2da041149a9863c5d5c1c3ac743ceb47e9f0d2 HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 168 gitlab # [ 243.977408] nixbot[2595]: {"ts": "2026-09-04T05:35:52.125Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/9e2da041149a9863c5d5c1c3ac743ceb47e9f0d2 \"HTTP/1.1 201 Created\""} gitlab # [ 244.009691] gitaly[926]: time="2026-09-04T05:35:52.158Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=968 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKK81SAWC77RGW3KKMB79 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-04T05:36:22.044" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:52.144" grpc.time_ms=13.791 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 244.027079] gitaly[926]: time="2026-09-04T05:35:52.176Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN3T9XQ25HYJHAB33V6C5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-04T05:36:22.075" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:52.175" grpc.time_ms=0.367 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 244.066869] gitaly[926]: time="2026-09-04T05:35:52.216Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEN3T9XQ25HYJHAB33V6C5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-04T05:36:02.200" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:52.210" grpc.time_ms=5.029 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 244.080165] gitaly[926]: time="2026-09-04T05:35:52.229Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKK81SAWC77RGW3KKMB79 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-04T05:36:22.005" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:52.206" grpc.time_ms=22.813 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 244.089142] sidekiq-cluster[2545]: {"severity":"WARN","time":"2026-09-04T05:35:52.238Z","class":"CreatePipelineWorker","correlation_id":"01M1NEKK81SAWC77RGW3KKMB79","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"9e2da041149a9863c5d5c1c3ac743ceb47e9f0d2","after":"a1b920324eb15e51aadde4b427f3f24773bfca94","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"a1b920324eb15e51aadde4b427f3f24773bfca94","ref":"refs/heads/master"},"retry":0} gitlab # [ 244.092832] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:52.241Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"855436918c2a6099ef166773","created_at":"2026-09-04T05:35:49.513Z","trace_propagation_headers":{"sentry-trace":"06f1c1278e884e529d8ea0deb2f12caf-6b1f5a68b644459e","baggage":"sentry-trace_id=06f1c1278e884e529d8ea0deb2f12caf,sentry-release=Unknown"},"correlation_id":"01M1NEKK81SAWC77RGW3KKMB79","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cafd6615b3319cfcb8412e780b08bb275d66861d10b134d021519aa235541a7e","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:49.516Z","job_size_bytes":467,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-855436918c2a6099ef166773: done: 0.539921 sec","job_status":"done","queue_duration_s":2.185138,"scheduling_latency_s":2.185164,"gitaly_calls":7,"gitaly_duration_s":0.111284,"redis_calls":33,"redis_duration_s":0.003533,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000543,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000669,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.001698,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000623,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.407868,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.539921,"completed_at":"2026-09-04T05:35:52.241Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016713,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 244.102793] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:52.243Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"09855c99-e6f6-4800-8fa9-02d929c22655\"}"],"class":"WebHookWorker","jid":"e81c8bc9b9c219e3361bcbba","created_at":"2026-09-04T05:35:49.538Z","trace_propagation_headers":{"sentry-trace":"06f1c1278e884e529d8ea0deb2f12caf-6b1f5a68b644459e","baggage":"sentry-trace_id=06f1c1278e884e529d8ea0deb2f12caf,sentry-release=Unknown"},"correlation_id":"01M1NEKK81SAWC77RGW3KKMB79","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:30b20e4638c5144ffe9586ea595f604c8bbc11153159f5c02b79488fe2505432","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:49.541Z","job_size_bytes":1749,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-e81c8bc9b9c219e3361bcbba: start","job_status":"start","queue_duration_s":2.701634,"scheduling_latency_s":2.701651} gitlab # [ 244.152796] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:52.271Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"09855c99-e6f6-4800-8fa9-02d929c22655\"}"],"class":"WebHookWorker","jid":"e81c8bc9b9c219e3361bcbba","created_at":"2026-09-04T05:35:49.538Z","trace_propagation_headers":{"sentry-trace":"06f1c1278e884e529d8ea0deb2f12caf-6b1f5a68b644459e","baggage":"sentry-trace_id=06f1c1278e884e529d8ea0deb2f12caf,sentry-release=Unknown"},"correlation_id":"01M1NEKK81SAWC77RGW3KKMB79","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:30b20e4638c5144ffe9586ea595f604c8bbc11153159f5c02b79488fe2505432","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:49.541Z","job_size_bytes":1749,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-e81c8bc9b9c219e3361bcbba: done: 0.027975 sec","job_status":"done","queue_duration_s":2.701634,"scheduling_latency_s":2.701651,"redis_calls":14,"redis_duration_s":0.002024,"redis_read_bytes":26,"redis_write_bytes":4685,"redis_queues_calls":2,"redis_queues_duration_s":0.000176,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001033,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000815,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.004986108999986527,"cpu_s":0.017065,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.027975,"completed_at":"2026-09-04T05:35:52.270Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001267,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.163615] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:52.272Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"165e6a37f66e8ec1db4d0e1f","created_at":"2026-09-04T05:35:49.921Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-35d35eb179944567","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:49.930Z","job_size_bytes":23,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-165e6a37f66e8ec1db4d0e1f: start","job_status":"start","queue_duration_s":2.342863,"scheduling_latency_s":2.342894} gitlab # [ 244.205687] nixbot[2595]: INFO: ::ffff:127.0.0.1:57820 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 244.206542] nixbot[2595]: {"ts": "2026-09-04T05:35:52.348Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/9e2da041149a9863c5d5c1c3ac743ceb47e9f0d2 \"HTTP/1.1 201 Created\""} gitlab # [ 244.207432] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:52 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 58 gitlab # [ 244.208100] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:52 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/9e2da041149a9863c5d5c1c3ac743ceb47e9f0d2 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 209 gitlab # [ 244.208577] gitaly[926]: time="2026-09-04T05:35:52.337Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=4 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEN3YQ4ZA9DEQ2TGTX1TE6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:52.326" grpc.time_ms=11.156 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 244.209304] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:52.332Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"165e6a37f66e8ec1db4d0e1f","created_at":"2026-09-04T05:35:49.921Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-35d35eb179944567","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:49.930Z","job_size_bytes":23,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-165e6a37f66e8ec1db4d0e1f: done: 0.05928 sec","job_status":"done","queue_duration_s":2.342863,"scheduling_latency_s":2.342894,"redis_calls":10,"redis_duration_s":0.0025830000000000002,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001378,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000891,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000314,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040957,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000225168,"duration_s":0.05928,"completed_at":"2026-09-04T05:35:52.332Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01274,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 244.258082] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:52.333Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"9b5559c9166cbdb599db56e7","created_at":"2026-09-04T05:35:50.133Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-35d35eb179944567","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:50.140Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-9b5559c9166cbdb599db56e7: start","job_status":"start","queue_duration_s":2.192485,"scheduling_latency_s":2.1925} gitlab # [ 244.258829] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:52.359Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"9b5559c9166cbdb599db56e7","created_at":"2026-09-04T05:35:50.133Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-35d35eb179944567","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:50.140Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-9b5559c9166cbdb599db56e7: done: 0.026824 sec","job_status":"done","queue_duration_s":2.192485,"scheduling_latency_s":2.1925,"redis_calls":10,"redis_duration_s":0.002183,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000408,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001339,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000436,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.002676597,"duration_s":0.026824,"completed_at":"2026-09-04T05:35:52.359Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003332,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.302565] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:52.364Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"fc417f49a1f141250a0a5784","created_at":"2026-09-04T05:35:50.155Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-35d35eb179944567","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:50.163Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-fc417f49a1f141250a0a5784: start","job_status":"start","queue_duration_s":2.201196,"scheduling_latency_s":2.201213} gitlab # [ 244.342698] gitaly[926]: time="2026-09-04T05:35:52.395Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN40ZZ5S8S97RYQTC9WYK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-04T05:36:22.294" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:52.394" grpc.time_ms=0.779 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 244.343647] nixbot[2595]: {"ts": "2026-09-04T05:35:52.459Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/9e2da041149a9863c5d5c1c3ac743ceb47e9f0d2 \"HTTP/1.1 201 Created\""} gitlab # [ 244.344147] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:52.389Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"fc417f49a1f141250a0a5784","created_at":"2026-09-04T05:35:50.155Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-35d35eb179944567","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:50.163Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-fc417f49a1f141250a0a5784: done: 0.024796 sec","job_status":"done","queue_duration_s":2.201196,"scheduling_latency_s":2.201213,"redis_calls":10,"redis_duration_s":0.0033220000000000003,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001128,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001815,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000379,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014743,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000546997,"duration_s":0.024796,"completed_at":"2026-09-04T05:35:52.389Z","load_balancing_strategy":"primary","db_duration_s":0.00408,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.401807] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:52.390Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"b1c5b055fdb6a25d2c36a4e0","created_at":"2026-09-04T05:35:50.180Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-35d35eb179944567","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:50.185Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-b1c5b055fdb6a25d2c36a4e0: start","job_status":"start","queue_duration_s":2.204664,"scheduling_latency_s":2.204678} gitlab # [ 244.402738] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:52 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/9e2da041149a9863c5d5c1c3ac743ceb47e9f0d2 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 105 gitlab # [ 244.403379] gitaly[926]: time="2026-09-04T05:35:52.421Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEN40ZZ5S8S97RYQTC9WYK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-04T05:36:02.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:52.415" grpc.time_ms=5.593 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 244.404090] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:52.414Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"b1c5b055fdb6a25d2c36a4e0","created_at":"2026-09-04T05:35:50.180Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-35d35eb179944567","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:50.185Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-b1c5b055fdb6a25d2c36a4e0: done: 0.024497 sec","job_status":"done","queue_duration_s":2.204664,"scheduling_latency_s":2.204678,"redis_calls":13,"redis_duration_s":0.0034790000000000003,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000854,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001539,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000519,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000567,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000303111,"duration_s":0.024497,"completed_at":"2026-09-04T05:35:52.414Z","load_balancing_strategy":"primary","db_duration_s":0.003052,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.460662] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:52.415Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"aabed8435f5ae259f9a234a2","created_at":"2026-09-04T05:35:50.186Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-35d35eb179944567","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:50.191Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-aabed8435f5ae259f9a234a2: start","job_status":"start","queue_duration_s":2.224652,"scheduling_latency_s":2.224665} gitlab # [ 244.461600] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:52.442Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"aabed8435f5ae259f9a234a2","created_at":"2026-09-04T05:35:50.186Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-35d35eb179944567","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:50.191Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-aabed8435f5ae259f9a234a2: done: 0.026661 sec","job_status":"done","queue_duration_s":2.224652,"scheduling_latency_s":2.224665,"redis_calls":10,"redis_duration_s":0.003167,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000297,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000964,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001906,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016993,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000236902,"duration_s":0.026661,"completed_at":"2026-09-04T05:35:52.442Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003712,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.539455] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:52.443Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"1d0d3d9fd72167e27c418751","created_at":"2026-09-04T05:35:50.191Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-35d35eb179944567","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:50.196Z","job_size_bytes":19,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-1d0d3d9fd72167e27c418751: start","job_status":"start","queue_duration_s":2.246682,"scheduling_latency_s":2.246696} gitlab # [ 244.540441] gitlab-workhorse[832]: time="2026-09-04T05:35:52Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 244.540873] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:52 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 240 gitlab # [ 244.541534] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:52 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 45 gitlab # [ 244.543339] gitlab-workhorse[832]: time="2026-09-04T05:35:52Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 244.544249] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:52 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 43 gitlab # [ 244.545164] gitaly[926]: time="2026-09-04T05:35:52.578Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN40K06SNT5091AHFQ2KF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:52.391" pid=926 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 244.545770] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:52.467Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"1d0d3d9fd72167e27c418751","created_at":"2026-09-04T05:35:50.191Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-35d35eb179944567","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:50.196Z","job_size_bytes":19,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-1d0d3d9fd72167e27c418751: done: 0.02413 sec","job_status":"done","queue_duration_s":2.246682,"scheduling_latency_s":2.246696,"redis_calls":11,"redis_duration_s":0.001794,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000273,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001068,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000453,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011921,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000196673,"duration_s":0.02413,"completed_at":"2026-09-04T05:35:52.467Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00489,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 244.588156] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:52.468Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"8485a57fab09bf8e3757866a","created_at":"2026-09-04T05:35:50.198Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-35d35eb179944567","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:50.201Z","job_size_bytes":66,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-8485a57fab09bf8e3757866a: start","job_status":"start","queue_duration_s":2.267047,"scheduling_latency_s":2.267061} gitlab # [ 244.589104] gitaly[926]: time="2026-09-04T05:35:52.578Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=185 command.spawn_token_fork_ms=181 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN40K06SNT5091AHFQ2KF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:52.391" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.04 seconds) gitlab # [ 244.602251] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:52.488Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"8485a57fab09bf8e3757866a","created_at":"2026-09-04T05:35:50.198Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-35d35eb179944567","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:50.201Z","job_size_bytes":66,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-8485a57fab09bf8e3757866a: done: 0.01955 sec","job_status":"done","queue_duration_s":2.267047,"scheduling_latency_s":2.267061,"redis_calls":10,"redis_duration_s":0.001464,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000258,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000786,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00042,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012745,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000222933,"duration_s":0.01955,"completed_at":"2026-09-04T05:35:52.488Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002816,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.648051] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:52.489Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"5e1bf1674fee240a8feb8f31","created_at":"2026-09-04T05:35:50.202Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-35d35eb179944567","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:50.205Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-5e1bf1674fee240a8feb8f31: start","job_status":"start","queue_duration_s":2.284109,"scheduling_latency_s":2.284121} gitlab # [ 244.658758] gitaly[926]: time="2026-09-04T05:35:52.578Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=185 command.spawn_token_fork_ms=181 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN40K06SNT5091AHFQ2KF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:52.391" grpc.time_ms=187.042 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 244.708436] nixbot[2595]: INFO: ::ffff:127.0.0.1:57824 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab # [ 244.709075] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:52.516Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"5e1bf1674fee240a8feb8f31","created_at":"2026-09-04T05:35:50.202Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-35d35eb179944567","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:50.205Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-5e1bf1674fee240a8feb8f31: done: 0.027468 sec","job_status":"done","queue_duration_s":2.284109,"scheduling_latency_s":2.284121,"redis_calls":10,"redis_duration_s":0.001395,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000264,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000883,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000248,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016887,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000278806,"duration_s":0.027468,"completed_at":"2026-09-04T05:35:52.516Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006649,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.751982] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:52.517Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"ec2d314b08e3d7b4a392f751","created_at":"2026-09-04T05:35:50.209Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-35d35eb179944567","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:50.212Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-ec2d314b08e3d7b4a392f751: start","job_status":"start","queue_duration_s":2.305313,"scheduling_latency_s":2.305325} gitlab # [ 244.752969] gitaly[926]: time="2026-09-04T05:35:52.643Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEN48P0Y41PPVBAM9ZR5DZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:52.636" grpc.time_ms=6.947 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 244.753671] gitaly[926]: time="2026-09-04T05:35:52.687Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN4A5EN9WSS4DCX850NYH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:52.680" pid=926 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 244.754268] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:52.539Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"ec2d314b08e3d7b4a392f751","created_at":"2026-09-04T05:35:50.209Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-35d35eb179944567","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:50.212Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-ec2d314b08e3d7b4a392f751: done: 0.021732 sec","job_status":"done","queue_duration_s":2.305313,"scheduling_latency_s":2.305325,"redis_calls":10,"redis_duration_s":0.001372,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000253,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00084,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000279,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012411,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000218184,"duration_s":0.021732,"completed_at":"2026-09-04T05:35:52.539Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005639,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.807260] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:52.540Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"3888f8f2b9ffe5634fe07615","created_at":"2026-09-04T05:35:50.212Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-35d35eb179944567","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:50.215Z","job_size_bytes":6,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-3888f8f2b9ffe5634fe07615: start","job_status":"start","queue_duration_s":2.324934,"scheduling_latency_s":2.324947} gitlab # [ 244.811932] gitaly[926]: time="2026-09-04T05:35:52.687Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN4A5EN9WSS4DCX850NYH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:52.680" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 244.824959] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:52.554Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"3888f8f2b9ffe5634fe07615","created_at":"2026-09-04T05:35:50.212Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-35d35eb179944567","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:50.215Z","job_size_bytes":6,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-3888f8f2b9ffe5634fe07615: done: 0.013969 sec","job_status":"done","queue_duration_s":2.324934,"scheduling_latency_s":2.324947,"redis_calls":10,"redis_duration_s":0.001403,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000257,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000888,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000258,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008382,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002288,"duration_s":0.013969,"completed_at":"2026-09-04T05:35:52.554Z","load_balancing_strategy":"primary","db_duration_s":0.001328,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.870783] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:52.555Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"e484128f98106715245be686","created_at":"2026-09-04T05:35:50.215Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-35d35eb179944567","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:50.218Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-e484128f98106715245be686: start","job_status":"start","queue_duration_s":2.336523,"scheduling_latency_s":2.336536} gitlab # [ 244.915685] gitaly[926]: time="2026-09-04T05:35:52.688Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN4A5EN9WSS4DCX850NYH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-04T05:35:52.680" grpc.time_ms=7.258 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 244.916578] nixbot[2595]: {"ts": "2026-09-04T05:35:53.014Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/a1b920324eb15e51aadde4b427f3f24773bfca94 \"HTTP/1.1 201 Created\""} gitlab # [ 244.917072] nixbot[2595]: {"ts": "2026-09-04T05:35:53.021Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-a1b920324eb1-5"} gitlab # [ 244.917558] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:52.577Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"e484128f98106715245be686","created_at":"2026-09-04T05:35:50.215Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-35d35eb179944567","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:50.218Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-e484128f98106715245be686: done: 0.022296 sec","job_status":"done","queue_duration_s":2.336523,"scheduling_latency_s":2.336536,"redis_calls":10,"redis_duration_s":0.001411,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000279,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000846,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000286,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01315,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000224051,"duration_s":0.022296,"completed_at":"2026-09-04T05:35:52.577Z","load_balancing_strategy":"primary","db_duration_s":0.005425,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.958791] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:52.578Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"53217690b9276eaa81598073","created_at":"2026-09-04T05:35:50.222Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-35d35eb179944567","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:50.226Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-53217690b9276eaa81598073: start","job_status":"start","queue_duration_s":2.352486,"scheduling_latency_s":2.3525} gitlab # [ 244.959428] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:53 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 185 gitlab # [ 244.959960] gitaly[926]: time="2026-09-04T05:35:52.782Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1NEKKR29D2WR8BPMMZD8SSH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-04T05:35:52.781" grpc.time_ms=0.301 pid=926 span.kind=server system=grpc gitlab # [ 244.966937] gitaly[926]: time="2026-09-04T05:35:52.804Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1NEKKR29D2WR8BPMMZD8SSH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-04T05:35:52.803" grpc.time_ms=0.351 pid=926 span.kind=server system=grpc gitlab # [ 245.011822] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:52.627Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"53217690b9276eaa81598073","created_at":"2026-09-04T05:35:50.222Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-35d35eb179944567","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:50.226Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-53217690b9276eaa81598073: done: 0.049491 sec","job_status":"done","queue_duration_s":2.352486,"scheduling_latency_s":2.3525,"redis_calls":14,"redis_duration_s":0.002916,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001209,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000874,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000833,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030335,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000227962,"duration_s":0.049491,"completed_at":"2026-09-04T05:35:52.627Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011788,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 245.017147] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:52.629Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"cdd300887b032bef20b3835e","created_at":"2026-09-04T05:35:50.326Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-35d35eb179944567","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:50.328Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-cdd300887b032bef20b3835e: start","job_status":"start","queue_duration_s":2.300047,"scheduling_latency_s":2.300062} gitlab # [ 245.065778] nixbot[2595]: {"ts": "2026-09-04T05:35:53.108Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-a1b920324eb1-5", "argv0": "bwrap"} gitlab # [ 245.066663] gitaly[926]: time="2026-09-04T05:35:52.861Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN4FV2R86H8W3612TA6FR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-04T05:36:22.760" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:52.861" grpc.time_ms=0.498 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.067576] gitaly[926]: time="2026-09-04T05:35:52.874Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEN4FV2R86H8W3612TA6FR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-04T05:36:22.768" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:52.868" grpc.time_ms=5.291 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.068287] gitaly[926]: time="2026-09-04T05:35:52.882Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEN4FV2R86H8W3612TA6FR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-04T05:36:02.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:52.877" grpc.time_ms=4.917 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.071631] gitaly[926]: time="2026-09-04T05:35:52.898Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN4FV2R86H8W3612TA6FR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-04T05:36:02.028" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:35:52.898" grpc.time_ms=0.198 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.081990] gitaly[926]: time="2026-09-04T05:35:52.905Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN4FV2R86H8W3612TA6FR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-04T05:36:02.030" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:52.900" grpc.time_ms=4.61 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.123796] gitaly[926]: time="2026-09-04T05:35:52.933Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1NEN4FV2R86H8W3612TA6FR grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-04T05:35:52.933" grpc.time_ms=0.268 pid=926 span.kind=server system=grpc gitlab # [ 245.124610] gitaly[926]: time="2026-09-04T05:35:52.953Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1NEN4FV2R86H8W3612TA6FR grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-04T05:35:52.952" grpc.time_ms=0.207 pid=926 span.kind=server system=grpc gitlab # [ 245.125210] gitaly[926]: time="2026-09-04T05:35:52.963Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN4FV2R86H8W3612TA6FR diskcache=8521662f-beed-450c-a482-519c1d3be8bd grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-04T05:36:02.036" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:35:52.906" pid=926 ref=refs/keep-around/a1b920324eb15e51aadde4b427f3f24773bfca94 remote_ip="2001:db8:1::1" revision=a1b920324eb15e51aadde4b427f3f24773bfca94 span.kind=server system=grpc username=root gitlab # [ 245.125879] gitaly[926]: time="2026-09-04T05:35:52.963Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=48 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5565 command.oublock=8 command.real_time_ms=49 command.spawn_token_fork_ms=2 command.system_time_ms=27 command.user_time_ms=21 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN4FV2R86H8W3612TA6FR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-04T05:36:02.036" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-04T05:35:52.906" grpc.time_ms=56.398 pid=926 ref=refs/keep-around/a1b920324eb15e51aadde4b427f3f24773bfca94 remote_ip="2001:db8:1::1" revision=a1b920324eb15e51aadde4b427f3f24773bfca94 span.kind=server system=grpc username=root gitlab # [ 245.137870] gitaly[926]: time="2026-09-04T05:35:52.966Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN4FV2R86H8W3612TA6FR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-04T05:36:22.865" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:52.966" grpc.time_ms=0.255 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.198941] gitaly[926]: time="2026-09-04T05:35:53.348Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1NEKKR29D2WR8BPMMZD8SSH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-04T05:35:53.347" grpc.time_ms=0.296 pid=926 span.kind=server system=grpc gitlab # [ 245.217609] gitaly[926]: time="2026-09-04T05:35:53.366Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKKR29D2WR8BPMMZD8SSH diskcache=1bb39612-6a20-4c9f-9b35-04eefac95df7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-04T05:36:22.491" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:52.691" pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.218260] gitaly[926]: time="2026-09-04T05:35:53.367Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=97 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=827392 command.minflt=7056 command.oublock=0 command.real_time_ms=667 command.spawn_token_fork_ms=0 command.system_time_ms=68 command.user_time_ms=29 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKKR29D2WR8BPMMZD8SSH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-04T05:36:22.491" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:52.691" grpc.time_ms=675.105 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.247701] nix-daemon[2823]: accepted connection from pid 3414, user nixbot gitlab # [ 245.282943] gitaly[926]: time="2026-09-04T05:35:53.432Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEKKR29D2WR8BPMMZD8SSH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-04T05:36:23.230" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:53.431" grpc.time_ms=0.978 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.313103] gitaly[926]: time="2026-09-04T05:35:53.462Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEKKR29D2WR8BPMMZD8SSH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-04T05:36:23.261" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:53.461" grpc.time_ms=0.774 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.325148] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.473Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"cdd300887b032bef20b3835e","created_at":"2026-09-04T05:35:50.326Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-35d35eb179944567","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:50.328Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2545,"job_status":"rescheduled","message":"PipelineProcessWorker JID-cdd300887b032bef20b3835e: rescheduled"} gitlab # [ 245.332403] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.481Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"cdd300887b032bef20b3835e","created_at":"2026-09-04T05:35:50.326Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-35d35eb179944567","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:50.328Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-cdd300887b032bef20b3835e: done: 0.852119 sec","job_status":"done","queue_duration_s":2.300047,"scheduling_latency_s":2.300062,"gitaly_calls":3,"gitaly_duration_s":0.704129,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.013807,"redis_read_bytes":152,"redis_write_bytes":18014,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000436,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000779,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.00297,"redis_queues_read_bytes":30,"redis_queues_write_bytes":12313,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.004975,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000164,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000887,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.00316,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.033,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.015,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.015,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.091439,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.814899659,"duration_s":0.852119,"completed_at":"2026-09-04T05:35:53.481Z","load_balancing_strategy":"primary","db_duration_s":0.043919,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 245.385078] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.482Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"b51f49fb4c251dfec4690cca","created_at":"2026-09-04T05:35:50.407Z","trace_propagation_headers":{"sentry-trace":"bc8395dfe7614f3283547c5e57875672-9f53ec7f644c491b","baggage":"sentry-trace_id=bc8395dfe7614f3283547c5e57875672,sentry-release=Unknown"},"correlation_id":"01M1NEKF552Z7PTSMVKD65DJA5","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:50.414Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-b51f49fb4c251dfec4690cca: start","job_status":"start","queue_duration_s":3.068185,"scheduling_latency_s":3.068211} gitlab # [ 245.386161] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.514Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"b51f49fb4c251dfec4690cca","created_at":"2026-09-04T05:35:50.407Z","trace_propagation_headers":{"sentry-trace":"bc8395dfe7614f3283547c5e57875672-9f53ec7f644c491b","baggage":"sentry-trace_id=bc8395dfe7614f3283547c5e57875672,sentry-release=Unknown"},"correlation_id":"01M1NEKF552Z7PTSMVKD65DJA5","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:50.414Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-b51f49fb4c251dfec4690cca: done: 0.032084 sec","job_status":"done","queue_duration_s":3.068185,"scheduling_latency_s":3.068211,"redis_calls":19,"redis_duration_s":0.0031070000000000004,"redis_read_bytes":22,"redis_write_bytes":4292,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00074,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000496,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2263,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001347,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000524,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020977,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.005307937,"duration_s":0.032084,"completed_at":"2026-09-04T05:35:53.514Z","load_balancing_strategy":"primary","db_duration_s":0.005992,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 245.430879] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.516Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","c880ae85d793e163729fad81"],"class":"WebHooks::LogExecutionWorker","jid":"f6709ed67882009c4aac861a","created_at":"2026-09-04T05:35:51.576Z","trace_propagation_headers":{"sentry-trace":"bc8395dfe7614f3283547c5e57875672-73d1820ad2d947ec","baggage":"sentry-trace_id=bc8395dfe7614f3283547c5e57875672,sentry-release=Unknown"},"correlation_id":"01M1NEKF552Z7PTSMVKD65DJA5","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6cdbc05ccd8b82a91245d46437b8940141a6be3a1cd0cad3750f999a483c4363","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:51.584Z","job_size_bytes":2593,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-f6709ed67882009c4aac861a: start","job_status":"start","queue_duration_s":1.931381,"scheduling_latency_s":1.931396} gitlab # [ 245.431563] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.546Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","c880ae85d793e163729fad81"],"class":"WebHooks::LogExecutionWorker","jid":"f6709ed67882009c4aac861a","created_at":"2026-09-04T05:35:51.576Z","trace_propagation_headers":{"sentry-trace":"bc8395dfe7614f3283547c5e57875672-73d1820ad2d947ec","baggage":"sentry-trace_id=bc8395dfe7614f3283547c5e57875672,sentry-release=Unknown"},"correlation_id":"01M1NEKF552Z7PTSMVKD65DJA5","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6cdbc05ccd8b82a91245d46437b8940141a6be3a1cd0cad3750f999a483c4363","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:51.584Z","job_size_bytes":2593,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-f6709ed67882009c4aac861a: done: 0.030649 sec","job_status":"done","queue_duration_s":1.931381,"scheduling_latency_s":1.931396,"redis_calls":10,"redis_duration_s":0.001931,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000375,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001147,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000409,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018909,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000260368,"duration_s":0.030649,"completed_at":"2026-09-04T05:35:53.546Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007209,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 245.471762] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.547Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","e81c8bc9b9c219e3361bcbba"],"class":"WebHooks::LogExecutionWorker","jid":"ec0bf87727e12fd21ec35374","created_at":"2026-09-04T05:35:52.262Z","trace_propagation_headers":{"sentry-trace":"06f1c1278e884e529d8ea0deb2f12caf-d80ff655212e4fa0","baggage":"sentry-trace_id=06f1c1278e884e529d8ea0deb2f12caf,sentry-release=Unknown"},"correlation_id":"01M1NEKK81SAWC77RGW3KKMB79","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:9b0a38fd8049d0666a9fdd2ebb67c8782b0efdbc96b4905e4297f515800bc5c6","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:52.263Z","job_size_bytes":2592,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-ec0bf87727e12fd21ec35374: start","job_status":"start","queue_duration_s":1.284122,"scheduling_latency_s":1.284137} gitlab # [ 245.475598] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.565Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","e81c8bc9b9c219e3361bcbba"],"class":"WebHooks::LogExecutionWorker","jid":"ec0bf87727e12fd21ec35374","created_at":"2026-09-04T05:35:52.262Z","trace_propagation_headers":{"sentry-trace":"06f1c1278e884e529d8ea0deb2f12caf-d80ff655212e4fa0","baggage":"sentry-trace_id=06f1c1278e884e529d8ea0deb2f12caf,sentry-release=Unknown"},"correlation_id":"01M1NEKK81SAWC77RGW3KKMB79","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:9b0a38fd8049d0666a9fdd2ebb67c8782b0efdbc96b4905e4297f515800bc5c6","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:52.263Z","job_size_bytes":2592,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-ec0bf87727e12fd21ec35374: done: 0.017165 sec","job_status":"done","queue_duration_s":1.284122,"scheduling_latency_s":1.284137,"redis_calls":7,"redis_duration_s":0.001614,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001197,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000417,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017165,"completed_at":"2026-09-04T05:35:53.565Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003898,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 245.527456] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.566Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"68aaa641d2aeb096ae86db67","created_at":"2026-09-04T05:35:52.684Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-03353fb222d94da3","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:52.686Z","job_size_bytes":23,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-68aaa641d2aeb096ae86db67: start","job_status":"start","queue_duration_s":0.880238,"scheduling_latency_s":0.880252} gitlab # [ 245.531602] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.620Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"68aaa641d2aeb096ae86db67","created_at":"2026-09-04T05:35:52.684Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-03353fb222d94da3","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:52.686Z","job_size_bytes":23,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-68aaa641d2aeb096ae86db67: done: 0.054062 sec","job_status":"done","queue_duration_s":0.880238,"scheduling_latency_s":0.880252,"redis_calls":7,"redis_duration_s":0.0023150000000000002,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001672,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000643,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038573,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.054062,"completed_at":"2026-09-04T05:35:53.620Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010043,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 245.576818] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.622Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"53891d0643a2c6a9cd25bc46","created_at":"2026-09-04T05:35:52.967Z","trace_propagation_headers":{"sentry-trace":"2fc9a5fc989b4e9caabf5b4a3c3b4934-81a3034247534e4d","baggage":"sentry-trace_id=2fc9a5fc989b4e9caabf5b4a3c3b4934,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEN4FV2R86H8W3612TA6FR","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:52.969Z","job_size_bytes":65,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-53891d0643a2c6a9cd25bc46: start","job_status":"start","queue_duration_s":0.652379,"scheduling_latency_s":0.652396} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94' gitlab # [ 245.621256] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.642Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"53891d0643a2c6a9cd25bc46","created_at":"2026-09-04T05:35:52.967Z","trace_propagation_headers":{"sentry-trace":"2fc9a5fc989b4e9caabf5b4a3c3b4934-81a3034247534e4d","baggage":"sentry-trace_id=2fc9a5fc989b4e9caabf5b4a3c3b4934,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEN4FV2R86H8W3612TA6FR","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:52.969Z","job_size_bytes":65,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-53891d0643a2c6a9cd25bc46: done: 0.020831 sec","job_status":"done","queue_duration_s":0.652379,"scheduling_latency_s":0.652396,"redis_calls":7,"redis_duration_s":0.001571,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001163,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000408,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011782,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020831,"completed_at":"2026-09-04T05:35:53.642Z","load_balancing_strategy":"primary","db_duration_s":0.005561,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 245.628101] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.643Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"ac7ee1394824b83a2d3ce23c","created_at":"2026-09-04T05:35:52.970Z","trace_propagation_headers":{"sentry-trace":"2fc9a5fc989b4e9caabf5b4a3c3b4934-81a3034247534e4d","baggage":"sentry-trace_id=2fc9a5fc989b4e9caabf5b4a3c3b4934,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEN4FV2R86H8W3612TA6FR","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:52.973Z","job_size_bytes":65,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-ac7ee1394824b83a2d3ce23c: start","job_status":"start","queue_duration_s":0.670888,"scheduling_latency_s":0.670903} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94', in 0.06 seconds) (finished: subtest: push delivers the webhook, in 51.73 seconds) subtest: eval and build post commit statuses gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a1b920324eb15e51aadde4b427f3f24773bfca94/statuses gitlab # [ 245.668994] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.707Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"ac7ee1394824b83a2d3ce23c","created_at":"2026-09-04T05:35:52.970Z","trace_propagation_headers":{"sentry-trace":"2fc9a5fc989b4e9caabf5b4a3c3b4934-81a3034247534e4d","baggage":"sentry-trace_id=2fc9a5fc989b4e9caabf5b4a3c3b4934,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEN4FV2R86H8W3612TA6FR","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:52.973Z","job_size_bytes":65,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-ac7ee1394824b83a2d3ce23c: done: 0.063085 sec","job_status":"done","queue_duration_s":0.670888,"scheduling_latency_s":0.670903,"redis_calls":10,"redis_duration_s":0.002521,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001285,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001236,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041411,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.063085,"completed_at":"2026-09-04T05:35:53.707Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016195,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 245.716093] gitaly[926]: time="2026-09-04T05:35:53.774Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEN4FV2R86H8W3612TA6FR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-04T05:36:23.573" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:53.773" grpc.time_ms=0.99 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.716872] nixbot[2595]: INFO: ::ffff:127.0.0.1:57840 - "GET /api/repos/gitlab/root/test-flake/builds?commit=a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 200 OK gitlab # [ 245.717592] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.709Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"f5eb8d9545b9fecdcf4dc7f3","created_at":"2026-09-04T05:35:53.003Z","trace_propagation_headers":{"sentry-trace":"2fc9a5fc989b4e9caabf5b4a3c3b4934-81a3034247534e4d","baggage":"sentry-trace_id=2fc9a5fc989b4e9caabf5b4a3c3b4934,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEN4FV2R86H8W3612TA6FR","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:53.005Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f5eb8d9545b9fecdcf4dc7f3: start","job_status":"start","queue_duration_s":0.70413,"scheduling_latency_s":0.704172} gitlab # [ 245.717751] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.816Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"f5eb8d9545b9fecdcf4dc7f3","created_at":"2026-09-04T05:35:53.003Z","trace_propagation_headers":{"sentry-trace":"2fc9a5fc989b4e9caabf5b4a3c3b4934-81a3034247534e4d","baggage":"sentry-trace_id=2fc9a5fc989b4e9caabf5b4a3c3b4934,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEN4FV2R86H8W3612TA6FR","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:53.005Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f5eb8d9545b9fecdcf4dc7f3: done: 0.107475 sec","job_status":"done","queue_duration_s":0.70413,"scheduling_latency_s":0.704172,"gitaly_calls":2,"gitaly_duration_s":0.00555,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.006647,"redis_read_bytes":80,"redis_write_bytes":5121,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000807,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001065,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000254,"redis_queues_read_bytes":3,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001847,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000194,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000532,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001141,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.071617,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.068779411,"duration_s":0.107475,"completed_at":"2026-09-04T05:35:53.816Z","load_balancing_strategy":"primary","db_duration_s":0.024865,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a1b920324eb15e51aadde4b427f3f24773bfca94/statuses, in 0.09 seconds) gitlab # [ 245.760869] gitaly[926]: time="2026-09-04T05:35:53.800Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN4FV2R86H8W3612TA6FR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-04T05:36:23.598" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:53.799" grpc.time_ms=0.833 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.771243] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.819Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"58b78905fc89fef19bb17cfd","created_at":"2026-09-04T05:35:53.391Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-03353fb222d94da3","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:53.393Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-58b78905fc89fef19bb17cfd: start","job_status":"start","queue_duration_s":0.425297,"scheduling_latency_s":0.42533} gitlab # [ 245.818031] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.848Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"58b78905fc89fef19bb17cfd","created_at":"2026-09-04T05:35:53.391Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-03353fb222d94da3","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:53.393Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-58b78905fc89fef19bb17cfd: done: 0.029853 sec","job_status":"done","queue_duration_s":0.425297,"scheduling_latency_s":0.42533,"redis_calls":7,"redis_duration_s":0.001627,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00119,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000437,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017783,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029853,"completed_at":"2026-09-04T05:35:53.848Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006283,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 245.862043] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.849Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"53ca167b64afb74b3d382b59","created_at":"2026-09-04T05:35:53.401Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-03353fb222d94da3","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:53.404Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-53ca167b64afb74b3d382b59: start","job_status":"start","queue_duration_s":0.445825,"scheduling_latency_s":0.445845} gitlab # [ 245.862722] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.873Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"53ca167b64afb74b3d382b59","created_at":"2026-09-04T05:35:53.401Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-03353fb222d94da3","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:53.404Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-53ca167b64afb74b3d382b59: done: 0.023925 sec","job_status":"done","queue_duration_s":0.445825,"scheduling_latency_s":0.445845,"redis_calls":7,"redis_duration_s":0.0016,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001147,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000453,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014226,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023925,"completed_at":"2026-09-04T05:35:53.873Z","load_balancing_strategy":"primary","db_duration_s":0.005688,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 245.910375] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:53 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a1b920324eb15e51aadde4b427f3f24773bfca94/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 62 gitlab # [ 245.911079] gitaly[926]: time="2026-09-04T05:35:53.935Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN5H2GWYJH5W1JJ1CNHKJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-04T05:36:23.834" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:53.934" grpc.time_ms=0.415 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.911693] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.875Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"c2449fc368a110cb9faf7962","created_at":"2026-09-04T05:35:53.409Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-03353fb222d94da3","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:53.411Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-c2449fc368a110cb9faf7962: start","job_status":"start","queue_duration_s":0.46381,"scheduling_latency_s":0.46384} gitlab # [ 245.912515] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.904Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"c2449fc368a110cb9faf7962","created_at":"2026-09-04T05:35:53.409Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-03353fb222d94da3","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:53.411Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-c2449fc368a110cb9faf7962: done: 0.028861 sec","job_status":"done","queue_duration_s":0.46381,"scheduling_latency_s":0.46384,"redis_calls":10,"redis_duration_s":0.0022400000000000002,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000798,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001043,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000399,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017229,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028861,"completed_at":"2026-09-04T05:35:53.904Z","load_balancing_strategy":"primary","db_duration_s":0.006594,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 245.956551] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.905Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"82de4532e80e88d7c2d6780f","created_at":"2026-09-04T05:35:53.412Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-03353fb222d94da3","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:53.414Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-82de4532e80e88d7c2d6780f: start","job_status":"start","queue_duration_s":0.490563,"scheduling_latency_s":0.490579} gitlab # [ 245.961886] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.928Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"82de4532e80e88d7c2d6780f","created_at":"2026-09-04T05:35:53.412Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-03353fb222d94da3","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:53.414Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-82de4532e80e88d7c2d6780f: done: 0.022699 sec","job_status":"done","queue_duration_s":0.490563,"scheduling_latency_s":0.490579,"redis_calls":7,"redis_duration_s":0.005512,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005257,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000255,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012001,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022699,"completed_at":"2026-09-04T05:35:53.928Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003552,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 246.024230] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.929Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"290df28646ff429faeeb0246","created_at":"2026-09-04T05:35:53.415Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-03353fb222d94da3","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:53.418Z","job_size_bytes":19,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-290df28646ff429faeeb0246: start","job_status":"start","queue_duration_s":0.511721,"scheduling_latency_s":0.511751} gitlab # [ 246.025063] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.947Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"290df28646ff429faeeb0246","created_at":"2026-09-04T05:35:53.415Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-03353fb222d94da3","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:53.418Z","job_size_bytes":19,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-290df28646ff429faeeb0246: done: 0.018177 sec","job_status":"done","queue_duration_s":0.511721,"scheduling_latency_s":0.511751,"redis_calls":8,"redis_duration_s":0.001311,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001056,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000255,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011145,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018177,"completed_at":"2026-09-04T05:35:53.947Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00243,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 246.073782] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.948Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"621846970e4829b3ac8192cf","created_at":"2026-09-04T05:35:53.418Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-03353fb222d94da3","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:53.419Z","job_size_bytes":66,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-621846970e4829b3ac8192cf: start","job_status":"start","queue_duration_s":0.528854,"scheduling_latency_s":0.528869} gitlab # [ 246.074585] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.962Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"621846970e4829b3ac8192cf","created_at":"2026-09-04T05:35:53.418Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-03353fb222d94da3","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:53.419Z","job_size_bytes":66,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-621846970e4829b3ac8192cf: done: 0.013475 sec","job_status":"done","queue_duration_s":0.528854,"scheduling_latency_s":0.528869,"redis_calls":7,"redis_duration_s":0.000914,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000665,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000249,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009423,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013475,"completed_at":"2026-09-04T05:35:53.962Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001517,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 246.113788] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.963Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"354ac39fae56599458fc5b10","created_at":"2026-09-04T05:35:53.420Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-03353fb222d94da3","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:53.421Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-354ac39fae56599458fc5b10: start","job_status":"start","queue_duration_s":0.541847,"scheduling_latency_s":0.541862} gitlab # [ 246.156733] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.982Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"354ac39fae56599458fc5b10","created_at":"2026-09-04T05:35:53.420Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-03353fb222d94da3","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:53.421Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-354ac39fae56599458fc5b10: done: 0.019667 sec","job_status":"done","queue_duration_s":0.541847,"scheduling_latency_s":0.541862,"redis_calls":7,"redis_duration_s":0.00207,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001802,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000268,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011654,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019667,"completed_at":"2026-09-04T05:35:53.982Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00432,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 246.198303] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.983Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"6bfb84bc7b81099618f2113a","created_at":"2026-09-04T05:35:53.421Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-03353fb222d94da3","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:53.422Z","job_size_bytes":6,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-6bfb84bc7b81099618f2113a: start","job_status":"start","queue_duration_s":0.561147,"scheduling_latency_s":0.561163} gitlab # [ 246.198968] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.992Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"6bfb84bc7b81099618f2113a","created_at":"2026-09-04T05:35:53.421Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-03353fb222d94da3","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:53.422Z","job_size_bytes":6,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-6bfb84bc7b81099618f2113a: done: 0.008716 sec","job_status":"done","queue_duration_s":0.561147,"scheduling_latency_s":0.561163,"redis_calls":7,"redis_duration_s":0.000879,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000637,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000242,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005495,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008716,"completed_at":"2026-09-04T05:35:53.992Z","load_balancing_strategy":"primary","db_duration_s":0.000261,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 246.245533] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:53.993Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"f9657cafe95c3e2df5905fe2","created_at":"2026-09-04T05:35:53.423Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-03353fb222d94da3","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:53.424Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-f9657cafe95c3e2df5905fe2: start","job_status":"start","queue_duration_s":0.569336,"scheduling_latency_s":0.569349} gitlab # [ 246.246260] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:54.006Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"f9657cafe95c3e2df5905fe2","created_at":"2026-09-04T05:35:53.423Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-03353fb222d94da3","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:53.424Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-f9657cafe95c3e2df5905fe2: done: 0.013264 sec","job_status":"done","queue_duration_s":0.569336,"scheduling_latency_s":0.569349,"redis_calls":7,"redis_duration_s":0.000871,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000625,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000246,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008869,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013264,"completed_at":"2026-09-04T05:35:54.006Z","load_balancing_strategy":"primary","db_duration_s":0.00186,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 246.286399] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:54.007Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"c6ed60c1055d0e4db2d1b0f1","created_at":"2026-09-04T05:35:53.473Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-03353fb222d94da3","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:53.475Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c6ed60c1055d0e4db2d1b0f1: start","job_status":"start","queue_duration_s":0.531567,"scheduling_latency_s":0.53158} gitlab # [ 246.287103] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:54.021Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"c6ed60c1055d0e4db2d1b0f1","created_at":"2026-09-04T05:35:53.473Z","trace_propagation_headers":{"sentry-trace":"36c644669f054751831179922d154ae7-03353fb222d94da3","baggage":"sentry-trace_id=36c644669f054751831179922d154ae7,sentry-release=Unknown"},"correlation_id":"01M1NEKKR29D2WR8BPMMZD8SSH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:53.475Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c6ed60c1055d0e4db2d1b0f1: done: 0.014545 sec","job_status":"done","queue_duration_s":0.531567,"scheduling_latency_s":0.53158,"redis_calls":8,"redis_duration_s":0.001049,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000808,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000241,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010062,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014545,"completed_at":"2026-09-04T05:35:54.021Z","load_balancing_strategy":"primary","db_duration_s":0.001965,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 246.334303] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:54.022Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"05ddce3efed688e643c93cb8","created_at":"2026-09-04T05:35:53.505Z","trace_propagation_headers":{"sentry-trace":"bc8395dfe7614f3283547c5e57875672-a0e48435c8f0452e","baggage":"sentry-trace_id=bc8395dfe7614f3283547c5e57875672,sentry-release=Unknown"},"correlation_id":"01M1NEKF552Z7PTSMVKD65DJA5","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:53.509Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-05ddce3efed688e643c93cb8: start","job_status":"start","queue_duration_s":0.513613,"scheduling_latency_s":0.513627} gitlab # [ 246.334968] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:54.075Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"05ddce3efed688e643c93cb8","created_at":"2026-09-04T05:35:53.505Z","trace_propagation_headers":{"sentry-trace":"bc8395dfe7614f3283547c5e57875672-a0e48435c8f0452e","baggage":"sentry-trace_id=bc8395dfe7614f3283547c5e57875672,sentry-release=Unknown"},"correlation_id":"01M1NEKF552Z7PTSMVKD65DJA5","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:53.509Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-05ddce3efed688e643c93cb8: done: 0.0528 sec","job_status":"done","queue_duration_s":0.513613,"scheduling_latency_s":0.513627,"redis_calls":11,"redis_duration_s":0.001518,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000255,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000976,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000287,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002112,"duration_s":0.0528,"completed_at":"2026-09-04T05:35:54.075Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01271,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 246.373891] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:54.076Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"b172aa300ea03059765a3aa3","created_at":"2026-09-04T05:35:53.762Z","trace_propagation_headers":{"sentry-trace":"2fc9a5fc989b4e9caabf5b4a3c3b4934-0713fbc8dc65410a","baggage":"sentry-trace_id=2fc9a5fc989b4e9caabf5b4a3c3b4934,sentry-release=Unknown"},"correlation_id":"01M1NEN4FV2R86H8W3612TA6FR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:53.763Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-b172aa300ea03059765a3aa3: start","job_status":"start","queue_duration_s":0.312999,"scheduling_latency_s":0.313013} gitlab # [ 246.416484] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:54.096Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"b172aa300ea03059765a3aa3","created_at":"2026-09-04T05:35:53.762Z","trace_propagation_headers":{"sentry-trace":"2fc9a5fc989b4e9caabf5b4a3c3b4934-0713fbc8dc65410a","baggage":"sentry-trace_id=2fc9a5fc989b4e9caabf5b4a3c3b4934,sentry-release=Unknown"},"correlation_id":"01M1NEN4FV2R86H8W3612TA6FR","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:53.763Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-b172aa300ea03059765a3aa3: done: 0.01991 sec","job_status":"done","queue_duration_s":0.312999,"scheduling_latency_s":0.313013,"redis_calls":7,"redis_duration_s":0.001155,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000699,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000456,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014459,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01991,"completed_at":"2026-09-04T05:35:54.096Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002627,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a1b920324eb15e51aadde4b427f3f24773bfca94/statuses gitlab # [ 246.925686] gitaly[926]: time="2026-09-04T05:35:55.074Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN6KZ7DESF7ED6PQVCVQ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-04T05:36:25.072" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:55.072" grpc.time_ms=0.874 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.957905] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:55 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a1b920324eb15e51aadde4b427f3f24773bfca94/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 99 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a1b920324eb15e51aadde4b427f3f24773bfca94/statuses, in 0.16 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a1b920324eb15e51aadde4b427f3f24773bfca94/statuses gitlab # [ 248.059939] gitaly[926]: time="2026-09-04T05:35:56.208Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN7QNQGD26P91C0AV2HYY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-04T05:36:26.206" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:56.207" grpc.time_ms=1.014 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.090185] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:56 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a1b920324eb15e51aadde4b427f3f24773bfca94/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 89 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a1b920324eb15e51aadde4b427f3f24773bfca94/statuses, in 0.13 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a1b920324eb15e51aadde4b427f3f24773bfca94/statuses gitlab # [ 249.232390] gitaly[926]: time="2026-09-04T05:35:57.380Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN8V3G5JP4K3ZME98DV46 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-04T05:36:27.279" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:57.380" grpc.time_ms=0.772 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 249.266067] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:57 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a1b920324eb15e51aadde4b427f3f24773bfca94/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 130 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a1b920324eb15e51aadde4b427f3f24773bfca94/statuses, in 0.17 seconds) gitlab # [ 249.672656] nix-daemon[2823]: accepted connection from pid 3454, user nixbot gitlab # [ 249.684718] nix-daemon[2823]: accepted connection from pid 3414, user nixbot gitlab # [ 249.764052] gitaly[926]: time="2026-09-04T05:35:57.913Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN9D8EK3TQEGE96JXZMJ5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-04T05:36:27.811" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:57.912" grpc.time_ms=0.736 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 249.815263] gitaly[926]: time="2026-09-04T05:35:57.964Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEN9D8EK3TQEGE96JXZMJ5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-04T05:36:07.088" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:57.958" grpc.time_ms=5.85 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 249.848313] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:57.992Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"c82a1cfd93f7bbb395a01e97","created_at":"2026-09-04T05:35:57.987Z","trace_propagation_headers":{"sentry-trace":"443070f5c148467da2f464eb2fc6de14-71de551db72c4b9e","baggage":"sentry-trace_id=443070f5c148467da2f464eb2fc6de14,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEN9D8EK3TQEGE96JXZMJ5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:57.990Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c82a1cfd93f7bbb395a01e97: start","job_status":"start","queue_duration_s":0.001456,"scheduling_latency_s":0.001498} gitlab # [ 249.885159] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:58 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 169 gitlab # [ 249.887741] nixbot[2595]: {"ts": "2026-09-04T05:35:58.036Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/a1b920324eb15e51aadde4b427f3f24773bfca94 \"HTTP/1.1 201 Created\""} gitlab # [ 249.966796] gitaly[926]: time="2026-09-04T05:35:58.115Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN9JR7VMSZXX5JDCKJWVH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-04T05:36:28.113" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:58.113" grpc.time_ms=1.903 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 249.992492] gitaly[926]: time="2026-09-04T05:35:58.141Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEN9JR7VMSZXX5JDCKJWVH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-04T05:36:08.036" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:58.136" grpc.time_ms=4.953 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.016417] gitaly[926]: time="2026-09-04T05:35:58.165Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1NEN9D8EK3TQEGE96JXZMJ5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-04T05:35:58.165" grpc.time_ms=0.505 pid=926 span.kind=server system=grpc gitlab # [ 250.025205] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:58 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 133 gitlab # [ 250.025983] nixbot[2595]: {"ts": "2026-09-04T05:35:58.175Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/a1b920324eb15e51aadde4b427f3f24773bfca94 \"HTTP/1.1 201 Created\""} gitlab # [ 250.055074] gitaly[926]: time="2026-09-04T05:35:58.204Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1NEN9D8EK3TQEGE96JXZMJ5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-04T05:35:58.203" grpc.time_ms=0.38 pid=926 span.kind=server system=grpc gitlab # [ 250.066784] gitaly[926]: time="2026-09-04T05:35:58.216Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN9Q98X0X7JASMRWYQAXB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-04T05:36:28.215" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:58.215" grpc.time_ms=0.486 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.082812] gitaly[926]: time="2026-09-04T05:35:58.232Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1NEN9D8EK3TQEGE96JXZMJ5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-04T05:35:58.231" grpc.time_ms=0.348 pid=926 span.kind=server system=grpc gitlab # [ 250.096167] gitaly[926]: time="2026-09-04T05:35:58.245Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN9D8EK3TQEGE96JXZMJ5 forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-04T05:36:28.098" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:58.099" operations="[repack_refs]" pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.102749] gitaly[926]: time="2026-09-04T05:35:58.252Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN9D8EK3TQEGE96JXZMJ5 diskcache=c786a914-a891-459b-aef7-f27c7658e8d4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-04T05:36:28.098" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:58.099" pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.103580] gitaly[926]: time="2026-09-04T05:35:58.252Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=76 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7273 command.oublock=0 command.real_time_ms=145 command.spawn_token_fork_ms=0 command.system_time_ms=49 command.user_time_ms=27 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN9D8EK3TQEGE96JXZMJ5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-04T05:36:28.098" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:58.099" grpc.time_ms=152.88 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.110551] gitaly[926]: time="2026-09-04T05:35:58.259Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1NEN9Q98X0X7JASMRWYQAXB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-04T05:36:08.153" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:58.253" grpc.time_ms=6.107 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.140225] gitaly[926]: time="2026-09-04T05:35:58.289Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN9D8EK3TQEGE96JXZMJ5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-04T05:36:28.098" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:58.099" pid=926 remote_ip="2001:db8:1::1" repository_info="{false {6 704 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-09-04 05:34:56.518524121 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root gitlab # [ 250.153315] gitaly[926]: time="2026-09-04T05:35:58.296Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1NEN9D8EK3TQEGE96JXZMJ5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-04T05:36:28.295" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:58.295" grpc.time_ms=0.516 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.153941] nixbot[2595]: {"ts": "2026-09-04T05:35:58.299Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/a1b920324eb15e51aadde4b427f3f24773bfca94 \"HTTP/1.1 201 Created\""} gitlab # [ 250.154585] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:58 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a1b920324eb15e51aadde4b427f3f24773bfca94 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 104 gitlab # [ 250.168981] gitaly[926]: time="2026-09-04T05:35:58.318Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN9D8EK3TQEGE96JXZMJ5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-04T05:36:28.098" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:58.099" pid=926 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root gitlab # [ 250.169904] gitaly[926]: time="2026-09-04T05:35:58.318Z" level=info msg="optimized repository" clean_stale_data_time=28.411432ms clean_work_trees_time="84.368µs" commit_graph_time="3.353µs" correlation_id=01M1NEN9D8EK3TQEGE96JXZMJ5 gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="3.632µs" pid=926 prune_time="3.352µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="3.911µs" storage_name=default system=housekeeping total_time=28.599166ms gitlab # [ 250.170456] gitaly[926]: time="2026-09-04T05:35:58.318Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN9D8EK3TQEGE96JXZMJ5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-04T05:36:28.098" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-04T05:35:58.099" pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.184430] gitaly[926]: time="2026-09-04T05:35:58.333Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN9D8EK3TQEGE96JXZMJ5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-04T05:36:28.332" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:58.333" grpc.time_ms=0.425 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.192027] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:58.341Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"a4fa1d53450437644e7dd957","created_at":"2026-09-04T05:35:58.339Z","trace_propagation_headers":{"sentry-trace":"443070f5c148467da2f464eb2fc6de14-b58e5df60f864476","baggage":"sentry-trace_id=443070f5c148467da2f464eb2fc6de14,sentry-release=Unknown"},"correlation_id":"01M1NEN9D8EK3TQEGE96JXZMJ5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"c82a1cfd93f7bbb395a01e97","job_size_bytes":3,"pid":2545,"job_status":"deduplicated","message":"PipelineProcessWorker JID-a4fa1d53450437644e7dd957: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60} gitlab # [ 250.193099] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:58.342Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"c82a1cfd93f7bbb395a01e97","created_at":"2026-09-04T05:35:57.987Z","trace_propagation_headers":{"sentry-trace":"443070f5c148467da2f464eb2fc6de14-71de551db72c4b9e","baggage":"sentry-trace_id=443070f5c148467da2f464eb2fc6de14,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEN9D8EK3TQEGE96JXZMJ5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:57.990Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2545,"job_status":"rescheduled","message":"PipelineProcessWorker JID-c82a1cfd93f7bbb395a01e97: rescheduled"} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a1b920324eb15e51aadde4b427f3f24773bfca94/statuses gitlab # [ 250.197409] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:58.346Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"c82a1cfd93f7bbb395a01e97","created_at":"2026-09-04T05:35:57.987Z","trace_propagation_headers":{"sentry-trace":"443070f5c148467da2f464eb2fc6de14-71de551db72c4b9e","baggage":"sentry-trace_id=443070f5c148467da2f464eb2fc6de14,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1NEN9D8EK3TQEGE96JXZMJ5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:57.990Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c82a1cfd93f7bbb395a01e97: done: 0.354303 sec","job_status":"done","queue_duration_s":0.001456,"scheduling_latency_s":0.001498,"gitaly_calls":3,"gitaly_duration_s":0.160027,"redis_calls":79,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.040931,"redis_read_bytes":239,"redis_write_bytes":21785,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000981,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":11,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001006,"redis_cache_read_bytes":20,"redis_cache_write_bytes":948,"redis_queues_calls":25,"redis_queues_duration_s":0.002973,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14470,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.004083,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000236,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000412,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.030259,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.044,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":16,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.006,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.009,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.109165,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.281317877,"duration_s":0.354303,"completed_at":"2026-09-04T05:35:58.346Z","load_balancing_strategy":"primary","db_duration_s":0.047772,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 250.246769] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:58.347Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"89d98018161a9bb4dc619d5e","created_at":"2026-09-04T05:35:58.092Z","trace_propagation_headers":{"sentry-trace":"443070f5c148467da2f464eb2fc6de14-b58e5df60f864476","baggage":"sentry-trace_id=443070f5c148467da2f464eb2fc6de14,sentry-release=Unknown"},"correlation_id":"01M1NEN9D8EK3TQEGE96JXZMJ5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:58.094Z","job_size_bytes":23,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-89d98018161a9bb4dc619d5e: start","job_status":"start","queue_duration_s":0.253003,"scheduling_latency_s":0.253019} gitlab # [ 250.292216] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:58.377Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"89d98018161a9bb4dc619d5e","created_at":"2026-09-04T05:35:58.092Z","trace_propagation_headers":{"sentry-trace":"443070f5c148467da2f464eb2fc6de14-b58e5df60f864476","baggage":"sentry-trace_id=443070f5c148467da2f464eb2fc6de14,sentry-release=Unknown"},"correlation_id":"01M1NEN9D8EK3TQEGE96JXZMJ5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:58.094Z","job_size_bytes":23,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-89d98018161a9bb4dc619d5e: done: 0.030502 sec","job_status":"done","queue_duration_s":0.253003,"scheduling_latency_s":0.253019,"redis_calls":7,"redis_duration_s":0.002228,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001956,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000272,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020119,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030502,"completed_at":"2026-09-04T05:35:58.377Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006582,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 250.293453] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:58.379Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"1f5d5f310f09b978002e8058","created_at":"2026-09-04T05:35:58.257Z","trace_propagation_headers":{"sentry-trace":"443070f5c148467da2f464eb2fc6de14-b58e5df60f864476","baggage":"sentry-trace_id=443070f5c148467da2f464eb2fc6de14,sentry-release=Unknown"},"correlation_id":"01M1NEN9D8EK3TQEGE96JXZMJ5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:58.259Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-1f5d5f310f09b978002e8058: start","job_status":"start","queue_duration_s":0.119602,"scheduling_latency_s":0.11962} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/a1b920324eb15e51aadde4b427f3f24773bfca94/statuses, in 0.10 seconds) (finished: subtest: eval and build post commit statuses, in 4.65 seconds) (finished: run the VM test script, in 300.15 seconds) gitlab # [ 250.333284] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:58.394Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"1f5d5f310f09b978002e8058","created_at":"2026-09-04T05:35:58.257Z","trace_propagation_headers":{"sentry-trace":"443070f5c148467da2f464eb2fc6de14-b58e5df60f864476","baggage":"sentry-trace_id=443070f5c148467da2f464eb2fc6de14,sentry-release=Unknown"},"correlation_id":"01M1NEN9D8EK3TQEGE96JXZMJ5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:58.259Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-1f5d5f310f09b978002e8058: done: 0.015996 sec","job_status":"done","queue_duration_s":0.119602,"scheduling_latency_s":0.11962,"redis_calls":7,"redis_duration_s":0.000943,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000709,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000234,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01061,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015996,"completed_at":"2026-09-04T05:35:58.394Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002585,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.381034] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:58.395Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"7ac49055f9299d09feba40c4","created_at":"2026-09-04T05:35:58.264Z","trace_propagation_headers":{"sentry-trace":"443070f5c148467da2f464eb2fc6de14-b58e5df60f864476","baggage":"sentry-trace_id=443070f5c148467da2f464eb2fc6de14,sentry-release=Unknown"},"correlation_id":"01M1NEN9D8EK3TQEGE96JXZMJ5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:58.267Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-7ac49055f9299d09feba40c4: start","job_status":"start","queue_duration_s":0.128895,"scheduling_latency_s":0.128908} gitlab # [ 250.381589] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:58.412Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"7ac49055f9299d09feba40c4","created_at":"2026-09-04T05:35:58.264Z","trace_propagation_headers":{"sentry-trace":"443070f5c148467da2f464eb2fc6de14-b58e5df60f864476","baggage":"sentry-trace_id=443070f5c148467da2f464eb2fc6de14,sentry-release=Unknown"},"correlation_id":"01M1NEN9D8EK3TQEGE96JXZMJ5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-09-04T05:35:58.267Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-7ac49055f9299d09feba40c4: done: 0.016336 sec","job_status":"done","queue_duration_s":0.128895,"scheduling_latency_s":0.128908,"redis_calls":7,"redis_duration_s":0.0009729999999999999,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000712,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000261,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016336,"completed_at":"2026-09-04T05:35:58.412Z","load_balancing_strategy":"primary","db_duration_s":0.002943,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.428646] gitlab-workhorse[832]: gitlab 2001:db8:1::1 - - [2026/09/04:05:35:58 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/a1b920324eb15e51aadde4b427f3f24773bfca94/statuses HTTP/1.1" 200 1334 "" "curl/8.21.0" 55 gitlab # [ 250.428972] gitaly[926]: time="2026-09-04T05:35:58.496Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1NEN9ZS3FR5X5DK79VCHMV1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-04T05:36:28.395" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-04T05:35:58.496" grpc.time_ms=0.405 pid=926 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root test script finished in 300.24s cleanup kill QemuMachine (pid 46) gitlab # [ 250.429611] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-09-04T05:35:58.413Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"4885fe43edf690b269a07830","created_at":"2026-09-04T05:35:58.272Z","trace_propagation_headers":{"sentry-trace":"443070f5c148467da2f464eb2fc6de14-b58e5df60f864476","baggage":"sentry-trace_id=443070f5c148467da2f464eb2fc6de14,sentry-release=Unknown"},"correlation_id":"01M1NEN9D8EK3TQEGE96JXZMJ5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueqemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/gxzhl7aaiid7zp3y47jqqiq7zg5mqpwp-python3-3.14.6/bin/python3.14) gitlab # u (finished: cleanup, in 1.41 seconds) additionally exposed symbols: gitlab, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh [] [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-09-04T05:35:01.676Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'a1b920324eb15e51aadde4b427f3f24773bfca94', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': [{'level': 'warning', 'message': "unable to download 'https://cache.nixos.org/…': Could not resolve hostname (6) Could not resolve host: cache.nixos.org", 'count': 1}], 'error': None, 'created_at': '2026-09-04T05:35:52.823481Z', 'started_at': '2026-09-04T05:35:53.018552Z', 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': None}] [{'id': 5, 'sha': 'a1b920324eb15e51aadde4b427f3f24773bfca94', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-04T05:35:52.999Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'a1b920324eb15e51aadde4b427f3f24773bfca94', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-04T05:35:52.999Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'a1b920324eb15e51aadde4b427f3f24773bfca94', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-04T05:35:52.999Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'a1b920324eb15e51aadde4b427f3f24773bfca94', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-04T05:35:52.999Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'a1b920324eb15e51aadde4b427f3f24773bfca94', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 4s (1 warning)', 'created_at': '2026-09-04T05:35:52.999Z', 'started_at': None, 'finished_at': '2026-09-04T05:35:57.980Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': 'a1b920324eb15e51aadde4b427f3f24773bfca94', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-09-04T05:35:58.158Z', 'started_at': None, 'finished_at': '2026-09-04T05:35:58.273Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]