tribuchet: building on jamie Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 3600 seconds run the VM test script start all VMs gitlab: starting vm gitlab: QEMU running (pid 13) gitlab # Disk image does not exist, creating the virtualisation disk image... gitlab # Formatting '/build/vm-state-gitlab/tmp.qI9BFOGcYa', fmt=raw size=1073741824 gitlab # mke2fs 1.47.3 (8-Jul-2025) gitlab # Discarding device blocks: 0/262144 done gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes gitlab # Filesystem UUID: 3746b198-1d37-4ca2-b2b1-bc0868ba6636 gitlab # Superblock backups stored on blocks: gitlab # 32768, 98304, 163840, 229376 gitlab # gitlab # Allocating group tables: 0/8 done gitlab # Writing inode tables: 0/8 done gitlab # Creating journal (8192 blocks): done gitlab # Writing superblocks and filesystem accounting information: 0/8 done gitlab # gitlab # Virtualisation disk image created. gitlab # Creating Nix store image... gitlab # Created Nix store image. (finished: start all VMs, in 34.57 seconds) gitlab: waiting for unit setup-gitlab.service gitlab: waiting for the VM to finish booting gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)... gitlab # gitlab # gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)... gitlab # gitlab # gitlab # Booting from ROM... gitlab # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.34 (nixbld@localhost) (gcc (GCC) 15.2.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Jun 1 15:51:08 UTC 2026 gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0nnagcx9pqqdc7g7rq24ryyql9wypmy8-nixos-system-gitlab-test/init regInfo=/nix/store/7mvqjl1nqav8zvlxn3zmqdq9ly0abf1a-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.000000] BIOS-provided physical RAM map: gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved gitlab # [ 0.000000] NX (Execute Disable) protection: active gitlab # [ 0.000000] APIC: Static calls initialized gitlab # [ 0.000000] SMBIOS 2.8 present. gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 gitlab # [ 0.000000] DMI: Memory slots populated: 1/1 gitlab # [ 0.000000] Hypervisor detected: KVM gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 gitlab # [ 0.000000] kvm-clock: using sched offset of 495092011 cycles gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns gitlab # [ 0.000005] tsc: Detected 2400.010 MHz processor gitlab # [ 0.000810] last_pfn = 0x1c0000 max_arch_pfn = 0x10000000000 gitlab # [ 0.000846] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs gitlab # [ 0.000849] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT gitlab # [ 0.000900] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.002752] found SMP MP-table at [mem 0x000f5470-0x000f547f] gitlab # [ 0.002763] Using GB pages for direct mapping gitlab # [ 0.002835] RAMDISK: [mem 0xbe4e0000-0xbffcffff] gitlab # [ 0.002842] ACPI: Early table checksum verification disabled gitlab # [ 0.002845] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) gitlab # [ 0.002849] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002853] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002859] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002861] ACPI: FACS 0x00000000BFFE0000 000040 gitlab # [ 0.002862] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002864] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002866] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002867] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2] gitlab # [ 0.002868] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e] gitlab # [ 0.002869] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] gitlab # [ 0.002869] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482] gitlab # [ 0.002869] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba] gitlab # [ 0.002870] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2] gitlab # [ 0.003344] No NUMA configuration found gitlab # [ 0.003345] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff] gitlab # [ 0.003349] NODE_DATA(0) allocated [mem 0x1bfffa780-0x1bffffcff] gitlab # [ 0.005789] Zone ranges: gitlab # [ 0.005790] DMA [mem 0x0000000000001000-0x0000000000ffffff] gitlab # [ 0.005791] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] gitlab # [ 0.005792] Normal [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.005793] Device empty gitlab # [ 0.005794] Movable zone start for each node gitlab # [ 0.005795] Early memory node ranges gitlab # [ 0.005795] node 0: [mem 0x0000000000001000-0x000000000009efff] gitlab # [ 0.005796] node 0: [mem 0x0000000000100000-0x00000000bffd7fff] gitlab # [ 0.005797] node 0: [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.005798] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff] gitlab # [ 0.005816] On node 0, zone DMA: 1 pages in unavailable ranges gitlab # [ 0.006075] On node 0, zone DMA: 97 pages in unavailable ranges gitlab # [ 0.112129] On node 0, zone Normal: 40 pages in unavailable ranges gitlab # [ 0.113130] ACPI: PM-Timer IO Port: 0x608 gitlab # [ 0.113147] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) gitlab # [ 0.113183] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 gitlab # [ 0.113186] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) gitlab # [ 0.113188] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) gitlab # [ 0.113189] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) gitlab # [ 0.113190] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) gitlab # [ 0.113191] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) gitlab # [ 0.113193] ACPI: Using ACPI (MADT) for SMP configuration information gitlab # [ 0.113194] ACPI: HPET id: 0x8086a201 base: 0xfed00000 gitlab # [ 0.113201] TSC deadline timer available gitlab # [ 0.113205] CPU topo: Max. logical packages: 1 gitlab # [ 0.113206] CPU topo: Max. logical dies: 1 gitlab # [ 0.113206] CPU topo: Max. dies per package: 1 gitlab # [ 0.113210] CPU topo: Max. threads per core: 1 gitlab # [ 0.113210] CPU topo: Num. cores per package: 4 gitlab # [ 0.113211] CPU topo: Num. threads per package: 4 gitlab # [ 0.113211] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs gitlab # [ 0.113236] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() gitlab # [ 0.113254] kvm-guest: KVM setup pv remote TLB flush gitlab # [ 0.113257] kvm-guest: setup PV sched yield gitlab # [ 0.113276] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] gitlab # [ 0.113277] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] gitlab # [ 0.113278] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff] gitlab # [ 0.113279] [mem 0xc0000000-0xfeffbfff] available for PCI devices gitlab # [ 0.113281] Booting paravirtualized kernel on KVM gitlab # [ 0.113285] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns gitlab # [ 0.117757] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 gitlab # [ 0.119969] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288 gitlab # [ 0.120018] kvm-guest: PV spinlocks enabled gitlab # [ 0.120020] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) gitlab # [ 0.120023] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0nnagcx9pqqdc7g7rq24ryyql9wypmy8-nixos-system-gitlab-test/init regInfo=/nix/store/7mvqjl1nqav8zvlxn3zmqdq9ly0abf1a-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.120117] Unknown kernel command line parameters "regInfo=/nix/store/7mvqjl1nqav8zvlxn3zmqdq9ly0abf1a-closure-info/registration", will be passed to user space. gitlab # [ 0.120129] random: crng init done gitlab # [ 0.120130] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes gitlab # [ 0.128867] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) gitlab # [ 0.133309] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) gitlab # [ 0.133384] software IO TLB: area num 4. gitlab # [ 0.206400] Fallback order for Node 0: 0 gitlab # [ 0.206408] Built 1 zonelists, mobility grouping on. Total pages: 1572726 gitlab # [ 0.206410] Policy zone: Normal gitlab # [ 0.208921] mem auto-init: stack:all(zero), heap alloc:on, heap free:off gitlab # [ 0.218019] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 gitlab # [ 0.231541] allocated 12582912 bytes of page_ext gitlab # [ 0.241265] ftrace: allocating 48565 entries in 192 pages gitlab # [ 0.241266] ftrace: allocated 192 pages with 2 groups gitlab # [ 0.242094] Dynamic Preempt: lazy gitlab # [ 0.242251] rcu: Preemptible hierarchical RCU implementation. gitlab # [ 0.242252] rcu: RCU event tracing is enabled. gitlab # [ 0.242252] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. gitlab # [ 0.242254] Trampoline variant of Tasks RCU enabled. gitlab # [ 0.242254] Rude variant of Tasks RCU enabled. gitlab # [ 0.242255] Tracing variant of Tasks RCU enabled. gitlab # [ 0.242255] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. gitlab # [ 0.242256] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 gitlab # [ 0.242286] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.242288] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.242288] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.246565] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16 gitlab # [ 0.246887] rcu: srcu_init: Setting srcu_struct sizes based on contention. gitlab # [ 0.247001] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) gitlab # [ 0.254225] Console: colour VGA+ 80x25 gitlab # [ 0.254229] printk: legacy console [tty0] enabled gitlab # [ 0.297848] printk: legacy console [ttyS0] enabled gitlab # [ 0.491328] ACPI: Core revision 20250807 gitlab # [ 0.492874] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns gitlab # [ 0.495630] APIC: Switch to symmetric I/O mode setup gitlab # [ 0.497324] x2apic enabled gitlab # [ 0.498531] APIC: Switched APIC routing to: physical x2apic gitlab # [ 0.500190] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask() gitlab # [ 0.502369] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself() gitlab # [ 0.505015] kvm-guest: setup PV IPIs gitlab # [ 0.507280] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 gitlab # [ 0.509081] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns gitlab # [ 0.512092] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400010) gitlab # [ 0.514093] x86/cpu: User Mode Instruction Prevention (UMIP) activated gitlab # [ 0.516144] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 gitlab # [ 0.517091] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 gitlab # [ 0.519095] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto gitlab # [ 0.521091] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl gitlab # [ 0.523092] Transient Scheduler Attacks: Mitigation: Clear CPU buffers gitlab # [ 0.525091] Spectre V2 : Mitigation: Enhanced / Automatic IBRS gitlab # [ 0.527090] Speculative Return Stack Overflow: Mitigation: Safe RET gitlab # [ 0.528091] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization gitlab # [ 0.529097] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier gitlab # [ 0.531091] active return thunk: srso_alias_return_thunk gitlab # [ 0.532129] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' gitlab # [ 0.534091] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' gitlab # [ 0.535091] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' gitlab # [ 0.537090] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' gitlab # [ 0.539090] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' gitlab # [ 0.540090] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' gitlab # [ 0.541091] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' gitlab # [ 0.542091] x86/fpu: Supporting XSAVE feature 0x800: 'Control-flow User registers' gitlab # [ 0.543090] x86/fpu: Supporting XSAVE feature 0x1000: 'Control-flow Kernel registers (KVM only)' gitlab # [ 0.545092] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 gitlab # [ 0.546090] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 gitlab # [ 0.548091] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 gitlab # [ 0.550091] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 gitlab # [ 0.551091] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 gitlab # [ 0.552091] x86/fpu: xstate_offset[11]: 2440, xstate_sizes[11]: 16 gitlab # [ 0.553090] x86/fpu: xstate_offset[12]: 2456, xstate_sizes[12]: 24 gitlab # [ 0.555090] x86/fpu: Enabled xstate features 0x1ae7, context size is 2480 bytes, using 'compacted' format. gitlab # [ 0.585235] Freeing SMP alternatives memory: 44K gitlab # [ 0.586093] pid_max: default: 32768 minimum: 301 gitlab # [ 0.588202] LSM: initializing lsm=capability,landlock,yama,bpf,ima gitlab # [ 0.589192] landlock: Up and running. gitlab # [ 0.590091] Yama: becoming mindful. gitlab # [ 0.591302] LSM support for eBPF active gitlab # [ 0.593138] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.594240] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.596934] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) gitlab # [ 0.597667] Performance Events: Fam17h+ core perfctr, AMD PMU driver. gitlab # [ 0.598094] ... version: 2 gitlab # [ 0.599092] ... bit width: 48 gitlab # [ 0.600107] ... generic counters: 6 gitlab # [ 0.601093] ... generic bitmap: 000000000000003f gitlab # [ 0.602092] ... fixed-purpose counters: 0 gitlab # [ 0.603093] ... fixed-purpose bitmap: 0000000000000000 gitlab # [ 0.604092] ... value mask: 0000ffffffffffff gitlab # [ 0.605092] ... max period: 00007fffffffffff gitlab # [ 0.606092] ... global_ctrl mask: 000000000000003f gitlab # [ 0.607213] signal: max sigframe size: 3376 gitlab # [ 0.608250] rcu: Hierarchical SRCU implementation. gitlab # [ 0.609097] rcu: Max phase no-delay instances is 400. gitlab # [ 0.610290] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level gitlab # [ 0.615509] smp: Bringing up secondary CPUs ... gitlab # [ 0.616489] smpboot: x86: Booting SMP configuration: gitlab # [ 0.617104] .... node #0, CPUs: #1 #2 #3 gitlab # [ 0.621210] smp: Brought up 1 node, 4 CPUs gitlab # [ 0.624096] smpboot: Total of 4 processors activated (19200.08 BogoMIPS) gitlab # [ 0.625667] Memory: 6013160K/6290904K available (17098K kernel code, 2721K rwdata, 13520K rodata, 3644K init, 3008K bss, 258292K reserved, 0K cma-reserved) gitlab # [ 0.628092] devtmpfs: initialized gitlab # [ 0.629266] x86/mm: Memory block size: 128MB gitlab # [ 0.633103] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns gitlab # [ 0.634141] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) gitlab # [ 0.635178] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). gitlab # [ 0.636208] pinctrl core: initialized pinctrl subsystem gitlab # [ 0.637473] PM: RTC time: 11:45:06, date: 2026-07-06 gitlab # [ 0.641073] NET: Registered PF_NETLINK/PF_ROUTE protocol family gitlab # [ 0.643437] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations gitlab # [ 0.644151] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations gitlab # [ 0.646263] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations gitlab # [ 0.647106] audit: initializing netlink subsys (disabled) gitlab # [ 0.648289] audit: type=2000 audit(1783338305.954:1): state=initialized audit_enabled=0 res=1 gitlab # [ 0.648299] thermal_sys: Registered thermal governor 'fair_share' gitlab # [ 0.650094] thermal_sys: Registered thermal governor 'bang_bang' gitlab # [ 0.651093] thermal_sys: Registered thermal governor 'step_wise' gitlab # [ 0.652093] thermal_sys: Registered thermal governor 'user_space' gitlab # [ 0.653093] thermal_sys: Registered thermal governor 'power_allocator' gitlab # [ 0.654112] cpuidle: using governor menu gitlab # [ 0.657371] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 gitlab # [ 0.658412] PCI: Using configuration type 1 for base access gitlab # [ 0.659093] PCI: Using configuration type 1 for extended access gitlab # [ 0.660246] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. gitlab # [ 0.663138] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages gitlab # [ 0.664093] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page gitlab # [ 0.665094] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages gitlab # [ 0.666092] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page gitlab # [ 0.671106] ACPI: Added _OSI(Module Device) gitlab # [ 0.674104] ACPI: Added _OSI(Processor Device) gitlab # [ 0.677101] ACPI: Added _OSI(Processor Aggregator Device) gitlab # [ 0.685991] ACPI: 1 ACPI AML tables successfully acquired and loaded gitlab # [ 0.695362] ACPI: Interpreter enabled gitlab # [ 0.696111] ACPI: PM: (supports S0 S3 S4 S5) gitlab # [ 0.697094] ACPI: Using IOAPIC for interrupt routing gitlab # [ 0.698129] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug gitlab # [ 0.699101] PCI: Using E820 reservations for host bridge windows gitlab # [ 0.703653] ACPI: Enabled 2 GPEs in block 00 to 0F gitlab # [ 0.720229] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) gitlab # [ 0.725115] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] gitlab # [ 0.732547] acpiphp: Slot [3] registered gitlab # [ 0.736243] acpiphp: Slot [4] registered gitlab # [ 0.739244] acpiphp: Slot [5] registered gitlab # [ 0.742238] acpiphp: Slot [6] registered gitlab # [ 0.745234] acpiphp: Slot [7] registered gitlab # [ 0.747211] acpiphp: Slot [8] registered gitlab # [ 0.750222] acpiphp: Slot [9] registered gitlab # [ 0.752210] acpiphp: Slot [10] registered gitlab # [ 0.755248] acpiphp: Slot [11] registered gitlab # [ 0.758214] acpiphp: Slot [12] registered gitlab # [ 0.760210] acpiphp: Slot [13] registered gitlab # [ 0.762151] acpiphp: Slot [14] registered gitlab # [ 0.763127] acpiphp: Slot [15] registered gitlab # [ 0.764126] acpiphp: Slot [16] registered gitlab # [ 0.766133] acpiphp: Slot [17] registered gitlab # [ 0.767141] acpiphp: Slot [18] registered gitlab # [ 0.768129] acpiphp: Slot [19] registered gitlab # [ 0.770480] acpiphp: Slot [20] registered gitlab # [ 0.773216] acpiphp: Slot [21] registered gitlab # [ 0.776186] acpiphp: Slot [22] registered gitlab # [ 0.779183] acpiphp: Slot [23] registered gitlab # [ 0.783183] acpiphp: Slot [24] registered gitlab # [ 0.786233] acpiphp: Slot [25] registered gitlab # [ 0.789205] acpiphp: Slot [26] registered gitlab # [ 0.792183] acpiphp: Slot [27] registered gitlab # [ 0.795213] acpiphp: Slot [28] registered gitlab # [ 0.798183] acpiphp: Slot [29] registered gitlab # [ 0.801184] acpiphp: Slot [30] registered gitlab # [ 0.805183] acpiphp: Slot [31] registered gitlab # [ 0.808259] PCI host bridge to bus 0000:00 gitlab # [ 0.811123] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] gitlab # [ 0.815100] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] gitlab # [ 0.819100] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] gitlab # [ 0.824101] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] gitlab # [ 0.828099] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 0.832107] pci_bus 0000:00: root bus resource [bus 00-ff] gitlab # [ 0.835631] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint gitlab # [ 0.839097] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint gitlab # [ 0.844101] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint gitlab # [ 0.855262] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f] gitlab # [ 0.859291] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk gitlab # [ 0.864102] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk gitlab # [ 0.869103] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk gitlab # [ 0.874105] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk gitlab # [ 0.878883] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint gitlab # [ 0.888228] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f] gitlab # [ 0.894099] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint gitlab # [ 0.900389] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI gitlab # [ 0.904119] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB gitlab # [ 0.907717] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint gitlab # [ 0.932296] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] gitlab # [ 0.938180] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] gitlab # [ 0.943251] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref] gitlab # [ 0.949191] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] gitlab # [ 0.980199] pci 0000:00:02.0: pci_fixup_video+0x0/0x100 took 30273 usecs gitlab # [ 0.987893] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 1.002137] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f] gitlab # [ 1.007138] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] gitlab # [ 1.011232] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref] gitlab # [ 1.016138] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] gitlab # [ 1.026963] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 1.039127] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f] gitlab # [ 1.042134] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] gitlab # [ 1.046248] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref] gitlab # [ 1.056988] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 1.071140] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf] gitlab # [ 1.075138] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] gitlab # [ 1.079244] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref] gitlab # [ 1.090293] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 1.105188] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df] gitlab # [ 1.110125] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] gitlab # [ 1.111146] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref] gitlab # [ 1.119759] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 1.135138] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f] gitlab # [ 1.139138] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] gitlab # [ 1.144234] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref] gitlab # [ 1.155306] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 1.168132] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff] gitlab # [ 1.172129] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] gitlab # [ 1.176221] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref] gitlab # [ 1.185231] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 1.199146] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff] gitlab # [ 1.203137] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] gitlab # [ 1.207233] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref] gitlab # [ 1.213136] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref] gitlab # [ 1.222298] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint gitlab # [ 1.233172] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] gitlab # [ 1.237204] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref] gitlab # [ 1.245098] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint gitlab # [ 1.257171] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f] gitlab # [ 1.261138] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] gitlab # [ 1.265233] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref] gitlab # [ 1.276449] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 1.289136] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f] gitlab # [ 1.293138] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff] gitlab # [ 1.298231] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref] gitlab # [ 1.316548] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 gitlab # [ 1.318313] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 gitlab # [ 1.322579] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 gitlab # [ 1.326652] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 gitlab # [ 1.331376] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 gitlab # [ 1.339393] iommu: Default domain type: Translated gitlab # [ 1.343102] iommu: DMA domain TLB invalidation policy: lazy mode gitlab # [ 1.348864] ACPI: bus type USB registered gitlab # [ 1.352384] usbcore: registered new interface driver usbfs gitlab # [ 1.356162] usbcore: registered new interface driver hub gitlab # [ 1.360196] usbcore: registered new device driver usb gitlab # [ 1.367240] NetLabel: Initializing gitlab # [ 1.369101] NetLabel: domain hash size = 128 gitlab # [ 1.373101] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO gitlab # [ 1.377314] NetLabel: unlabeled traffic allowed by default gitlab # [ 1.381138] PCI: Using ACPI for IRQ routing gitlab # [ 1.386267] pci 0000:00:02.0: vgaarb: setting as boot VGA device gitlab # [ 1.387088] pci 0000:00:02.0: vgaarb: bridge control possible gitlab # [ 1.387088] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none gitlab # [ 1.399106] vgaarb: loaded gitlab # [ 1.401552] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 gitlab # [ 1.404103] hpet0: 3 comparators, 64-bit 100.000000 MHz counter gitlab # [ 1.414090] clocksource: Switched to clocksource kvm-clock gitlab # [ 1.422651] VFS: Disk quotas dquot_6.6.0 gitlab # [ 1.425832] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) gitlab # [ 1.431867] pnp: PnP ACPI init gitlab # [ 1.436169] pnp: PnP ACPI: found 6 devices gitlab # [ 1.452921] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns gitlab # [ 1.459270] clocksource: Switched to clocksource acpi_pm gitlab # [ 1.463725] NET: Registered PF_INET protocol family gitlab # [ 1.470735] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) gitlab # [ 1.502661] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) gitlab # [ 1.505163] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) gitlab # [ 1.507530] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) gitlab # [ 1.512310] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) gitlab # [ 1.514551] TCP: Hash tables configured (established 65536 bind 65536) gitlab # [ 1.516545] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) gitlab # [ 1.519068] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 1.521430] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 1.523675] NET: Registered PF_UNIX/PF_LOCAL protocol family gitlab # [ 1.525445] NET: Registered PF_XDP protocol family gitlab # [ 1.526926] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] gitlab # [ 1.528794] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] gitlab # [ 1.530607] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] gitlab # [ 1.532596] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] gitlab # [ 1.534580] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 1.536897] pci 0000:00:01.0: PIIX3: Enabling Passive Release gitlab # [ 1.538669] pci 0000:00:00.0: Limiting direct PCI/PCI transfers gitlab # [ 1.541936] ACPI: \_SB_.LNKD: Enabled at IRQ 11 gitlab # [ 1.545518] PCI: CLS 0 bytes, default 64 gitlab # [ 1.546885] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) gitlab # [ 1.547397] Trying to unpack rootfs image as initramfs... gitlab # [ 1.548799] software IO TLB: mapped [mem 0x00000000ba4e0000-0x00000000be4e0000] (64MB) gitlab # [ 1.548925] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns gitlab # [ 1.590344] Initialise system trusted keyrings gitlab # [ 1.594586] workingset: timestamp_bits=40 max_order=21 bucket_order=0 gitlab # [ 1.624656] Key type asymmetric registered gitlab # [ 1.625965] Asymmetric key parser 'x509' registered gitlab # [ 1.627639] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) gitlab # [ 1.631990] io scheduler mq-deadline registered gitlab # [ 1.634849] io scheduler kyber registered gitlab # [ 1.640054] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled gitlab # [ 1.642420] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A gitlab # [ 1.647442] Linux agpgart interface v0.103 gitlab # [ 1.650848] ACPI: bus type drm_connector registered gitlab # [ 1.657358] usbcore: registered new interface driver usbserial_generic gitlab # [ 1.659283] usbserial: USB Serial support registered for generic gitlab # [ 1.661045] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. gitlab # [ 1.661045] Please enable it if your BIOS has the CPPC option. gitlab # [ 1.664957] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled gitlab # [ 1.667370] drop_monitor: Initializing network drop monitor service gitlab # [ 1.669459] NET: Registered PF_INET6 protocol family gitlab # [ 1.672593] Segment Routing with IPv6 gitlab # [ 1.675687] In-situ OAM (IOAM) with IPv6 gitlab # [ 1.682331] IPI shorthand broadcast: enabled gitlab # [ 1.695924] sched_clock: Marking stable (1441073053, 254193365)->(1771645978, -76379560) gitlab # [ 1.703335] registered taskstats version 1 gitlab # [ 1.707615] Loading compiled-in X.509 certificates gitlab # [ 1.730073] Demotion targets for Node 0: null gitlab # [ 1.731860] Key type .fscrypt registered gitlab # [ 1.733562] Key type fscrypt-provisioning registered gitlab # [ 1.735296] ima: No TPM chip found, activating TPM-bypass! gitlab # [ 1.736915] ima: Allocated hash algorithm: sha1 gitlab # [ 1.738385] ima: No architecture policies found gitlab # [ 1.740166] PM: Magic number: 2:857:781 gitlab # [ 1.742541] RAS: Correctable Errors collector initialized. gitlab # [ 1.747883] clk: Disabling unused clocks gitlab # [ 1.749128] PM: genpd: Disabling unused power domains gitlab # [ 1.846355] Freeing initrd memory: 27584K gitlab # [ 1.854841] Freeing unused decrypted memory: 2028K gitlab # [ 1.863483] Freeing unused kernel image (initmem) memory: 3644K gitlab # [ 1.868158] Write protecting the kernel read-only data: 32768k gitlab # [ 1.875609] Freeing unused kernel image (text/rodata gap) memory: 1332K gitlab # [ 1.881927] Freeing unused kernel image (rodata/data gap) memory: 816K gitlab # [ 1.945286] x86/mm: Checked W+X mappings: passed, no W+X pages found. gitlab # [ 1.947269] Run /init as init process gitlab # [ 1.961517] systemd[1]: Inserted module 'autofs4' gitlab # [ 1.989657] fuse: init (API version 7.45) gitlab # [ 1.999658] ACPI: \_SB_.LNKC: Enabled at IRQ 10 gitlab # [ 2.009029] ACPI: \_SB_.LNKA: Enabled at IRQ 10 gitlab # [ 2.013820] ACPI: \_SB_.LNKB: Enabled at IRQ 11 gitlab # [ 2.113928] systemd[1]: Successfully made /usr/ read-only. gitlab # [ 2.454766] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 2.476017] systemd[1]: Detected virtualization kvm. gitlab # [ 2.480008] systemd[1]: Detected architecture x86-64. gitlab # [ 2.483957] systemd[1]: Running in initrd. gitlab # [ 2.488144] systemd[1]: Initializing machine ID from random generator. gitlab # [ 2.493413] systemd[1]: Hostname set to . gitlab # [ 2.571691] systemd[1]: Queued start job for default target Initrd Default Target. gitlab # [ 2.701477] systemd[1]: Created slice Slice /system/modprobe. gitlab # [ 2.706531] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 2.712540] systemd[1]: Expecting device /dev/disk/by-label/nix-store... gitlab # [ 2.717590] systemd[1]: Expecting device /dev/disk/by-label/nixos... gitlab # [ 2.722475] systemd[1]: Reached target Path Units. gitlab # [ 2.726455] systemd[1]: Reached target Slice Units. gitlab # [ 2.729742] systemd[1]: Reached target Swaps. gitlab # [ 2.732623] systemd[1]: Reached target Timer Units. gitlab # [ 2.735977] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 2.740130] systemd[1]: Listening on Journal Socket (/dev/log). gitlab # [ 2.744134] systemd[1]: Listening on Journal Sockets. gitlab # [ 2.746663] systemd[1]: Listening on udev Control Socket. gitlab # [ 2.748456] systemd[1]: Listening on udev Kernel Socket. gitlab # [ 2.750126] systemd[1]: Reached target Socket Units. gitlab # [ 2.756459] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 2.762546] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 2.776803] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 2.782130] systemd[1]: Starting Journal Service... gitlab # [ 2.793316] systemd[1]: Starting Load Kernel Modules... gitlab # [ 2.797296] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 2.802231] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 2.813120] systemd-journald[153]: Collecting audit messages is disabled. gitlab # [ 2.818222] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 2.824659] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 2.834398] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 2.837626] netfs: FS-Cache loaded gitlab # [ 2.840317] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config gitlab # [ 2.847050] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. gitlab # [ 2.850943] 9pnet: Installing 9P2000 support gitlab # [ 2.851095] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev gitlab # [ 2.858358] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 2.865887] systemd[1]: Finished Load Kernel Modules. gitlab # [ 2.868363] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 2.872591] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 2.879298] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 2.884401] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 2.892319] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 2.900317] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 2.904710] systemd[1]: Started Journal Service. gitlab # [ 2.650695] systemd-modules-load[156]: Using 4 probe threads gitlab # [ 2.654169] systemd-modules-load[156]: Inserted module 'virtio_balloon' gitlab # [ 2.655963] systemd-modules-load[156]: Inserted module 'dm_mod' gitlab # [ 2.657635] systemd-modules-load[156]: Inserted module 'virtio_gpu' gitlab # [ 2.659334] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 2.665745] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 2.667413] systemd[1]: Reached target Local File Systems. gitlab # [ 2.671203] systemd[1]: Starting Create System Files and Directories... gitlab # [ 2.685411] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 2.694542] systemd[1]: Finished Create System Files and Directories. gitlab # [ 2.707720] systemd-udevd[194]: Using default interface naming scheme 'v260'. gitlab # [ 2.724872] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 2.749910] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 2.751582] systemd[1]: Reached target System Initialization. gitlab # [ 2.753094] systemd[1]: Reached target Basic System. gitlab # [ 3.153542] virtio_blk virtio4: 4/0/0 default/read/poll queues gitlab # [ 3.159455] uhci_hcd 0000:00:01.2: UHCI Host Controller gitlab # [ 3.162495] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) gitlab # [ 3.163385] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1 gitlab # [ 3.171405] virtio_blk virtio5: 4/0/0 default/read/poll queues gitlab # [ 3.173655] uhci_hcd 0000:00:01.2: detected 2 ports gitlab # [ 3.175272] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140 gitlab # [ 3.177771] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 gitlab # [ 3.177910] SCSI subsystem initialized gitlab # [ 3.178250] virtio_blk virtio5: [vdb] 11450744 512-byte logical blocks (5.86 GB/5.46 GiB) gitlab # [ 3.179995] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 gitlab # [ 3.185484] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 gitlab # [ 3.187222] usb usb1: Product: UHCI Host Controller gitlab # [ 3.188449] usb usb1: Manufacturer: Linux 6.18.34 uhci_hcd gitlab # [ 3.190042] usb usb1: SerialNumber: 0000:00:01.2 gitlab # [ 3.191467] hub 1-0:1.0: USB hub found gitlab # [ 3.191608] serio: i8042 KBD port at 0x60,0x64 irq 1 gitlab # [ 3.191792] hub 1-0:1.0: 2 ports detected gitlab # [ 3.192065] serio: i8042 AUX port at 0x60,0x64 irq 12 gitlab # [ 3.253917] scsi host0: ata_piix gitlab # [ 3.003480] systemd[1]: Starting Virtual Console Setup... gitlab # [ 3.259406] scsi host1: ata_piix gitlab # [ 3.262711] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0 gitlab # [ 3.264767] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0 gitlab # [ 3.026665] (udev-worker)[202]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 3.029301] systemd-vconsole-setup[230]: Configuration of first virtual console was skipped, ignorin[ 3.284757] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 gitlab # g remaining ones. gitlab # [ 3.033618] systemd[1]: Found device /dev/disk/by-label/nixos. gitlab # [ 3.035291] (udev-worker)[220]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 3.038989] (udev-worker)[220]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 3.041109] systemd[1]: Finished Virtual Console Setup. gitlab # [ 3.045362] systemd[1]: Found device /dev/disk/by-label/nix-store. gitlab # [ 3.047112] systemd[1]: Reached target Initrd Root Device. gitlab # [ 3.048529] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... gitlab # [ 3.089050] systemd-fsck[244]: nixos: clean, 12/65536 files, 13019/262144 blocks gitlab # [ 3.096045] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. gitlab # [ 3.413433] usb 1-1: new full-speed USB device number 2 using uhci_hcd gitlab # [ 3.420939] ata2: found unknown device (class 0) gitlab # [ 3.426096] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 gitlab # [ 3.433336] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 gitlab # [ 3.499354] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray gitlab # [ 3.530550] cdrom: Uniform CD-ROM driver Revision: 3.20 gitlab # [ 3.586688] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 gitlab # [ 3.591464] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 gitlab # [ 3.595712] usb 1-1: Product: QEMU USB Tablet gitlab # [ 3.598551] usb 1-1: Manufacturer: QEMU gitlab # [ 3.601072] usb 1-1: SerialNumber: 28754-0000:00:01.2-1 gitlab # [ 3.639741] hid: raw HID events driver (C) Jiri Kosina gitlab # [ 3.655511] usbcore: registered new interface driver usbhid gitlab # [ 3.659431] usbhid: USB HID core driver gitlab # [ 3.674371] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 gitlab # [ 3.680817] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0 gitlab # [ 3.564565] systemd[1]: Mounting /sysroot... gitlab # [ 4.000995] EXT4-fs (vda): mounted filesystem 3746b198-1d37-4ca2-b2b1-bc0868ba6636 r/w with ordered data mode. Quota mode: none. gitlab # [ 3.752153] systemd[1]: Mounted /sysroot. gitlab # [ 3.754702] systemd[1]: Reached target Initrd Root File System. gitlab # [ 3.757770] systemd[1]: Mounting /sysroot/nix/.ro-store... gitlab # [ 3.760697] systemd[1]: Mounting /sysroot/nix/.rw-store... gitlab # [ 3.763600] systemd[1]: Mounting /sysroot/run... gitlab # [ 3.770233] systemd[1]: Mounting /sysroot/tmp/shared... gitlab # [ 3.775885] systemd[1]: Mounting /sysroot/tmp/xchg... gitlab # [ 3.778893] systemd[1]: Starting Mountpoints Configured in the Real Root... gitlab # [ 3.785484] systemd[1]: Mounted /sysroot/nix/.rw-store. gitlab # [ 3.788211] systemd[1]: Mounted /sysroot/run. gitlab # [ 3.789853] systemd-sysroot-fstab-check[271]: /sysroot should be mounted in the initrd, will request daemon-reload. gitlab # [ 4.048839] erofs (device vdb): mounted with root inode @ nid 128. gitlab # [ 3.797167] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.798863] systemd[1]: Mounted /sysroot/nix/[ 4.053152] 9p: Installing v9fs 9p2000 file system support gitlab # .ro-store. gitlab # [ 3.801539] systemd[1]: Reload requested from client PID 271 ('systemd-sysroot') (unit initrd-parse-etc.service)... gitlab # [ 3.805206] systemd[1]: Reloading... gitlab # [ 4.042160] systemd[1]: Reloading finished in 241 ms. gitlab # [ 4.101725] systemd-sysroot-fstab-check[271]: Requesting initrd-fs.target/start/replace... gitlab # [ 4.260178] systemd[1]: Mounted /sysroot/tmp/shared. gitlab # [ 4.263429] systemd[1]: Mounted /sysroot/tmp/xchg. gitlab # [ 4.266451] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 4.273215] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 4.292947] systemd-sysroot-fstab-check[271]: Requesting swap.target/start/replace... gitlab # [ 4.303774] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 4.307348] systemd[1]: initrd-parse-etc.service: Deactivated successfully. gitlab # [ 4.312805] systemd[1]: Finished Mountpoints Configured in the Real Root. gitlab # [ 4.316918] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. gitlab # [ 4.331871] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 4.336715] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 4.563848] systemd[1]: Mounting /sysroot/nix/store... gitlab # [ 4.613249] systemd[1]: Mounted /sysroot/nix/store. gitlab # [ 4.616462] systemd[1]: Reached target Initrd File Systems. gitlab # [ 4.621979] systemd[1]: Starting Find NixOS closure... gitlab # [ 4.627881] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... gitlab # [ 4.659494] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. gitlab # [ 4.664178] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. gitlab # [ 4.679977] systemd[1]: Finished Find NixOS closure. gitlab # [ 4.685500] systemd[1]: Reached target Initrd Default Target. gitlab # [ 4.693188] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... gitlab # [ 4.723411] systemd[1]: Stopped target Initrd Default Target. gitlab # [ 4.726867] systemd[1]: Stopped target Basic System. gitlab # [ 4.729978] systemd[1]: Stopped target Initrd Root Device. gitlab # [ 4.733400] systemd[1]: Stopped target Path Units. gitlab # [ 4.736403] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. gitlab # [ 4.740779] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. gitlab # [ 4.745293] systemd[1]: Stopped target Slice Units. gitlab # [ 4.748389] systemd[1]: Stopped target Socket Units. gitlab # [ 4.751151] systemd[1]: Stopped target System Initialization. gitlab # [ 4.754674] systemd[1]: Stopped target Swaps. gitlab # [ 4.756838] systemd[1]: Stopped target Timer Units. gitlab # [ 4.759257] systemd[1]: dbus.socket: Deactivated successfully. gitlab # [ 4.761949] systemd[1]: Closed D-Bus System Message Bus Socket. gitlab # [ 4.764693] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. gitlab # [ 4.767905] systemd[1]: Stopped Find NixOS closure. gitlab # [ 4.769816] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 4.774306] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 4.776055] systemd[1]: systemd-sysctl.service: Deactivated successfully. gitlab # [ 4.777753] systemd[1]: Stopped Apply Kernel Variables. gitlab # [ 4.780234] systemd[1]: systemd-modules-load.service: Deactivated successfully. gitlab # [ 4.781952] systemd[1]: Stopped Load Kernel Modules. gitlab # [ 4.787966] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. gitlab # [ 4.789872] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. gitlab # [ 4.794851] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. gitlab # [ 4.796595] systemd[1]: Stopped Create System Files and Directories. gitlab # [ 4.801090] systemd[1]: Stopped target Local File Systems. gitlab # [ 4.802466] systemd[1]: Stopped target Preparation for Local File Systems. gitlab # [ 4.804062] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. gitlab # [ 4.805761] systemd[1]: Stopped Coldplug All udev Devices. gitlab # [ 4.811243] systemd[1]: Stopping Rule-based Manager for Device Events and Files... gitlab # [ 4.814404] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 4.816871] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 4.825366] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 4.829159] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 4.832608] systemd[1]: systemd-udevd.service: Deactivated successfully. gitlab # [ 4.836055] systemd[1]: Stopped Rule-based Manager for Device Events and Files. gitlab # [ 4.837725] systemd[1]: systemd-udevd.service: Consumed 1.106s CPU time over 2.147s wall clock time, 25.2M memory peak. gitlab # [ 4.848444] systemd[1]: initrd-cleanup.service: Deactivated successfully. gitlab # [ 4.853525] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. gitlab # [ 4.860609] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. gitlab # [ 4.862347] systemd[1]: Closed udev Control Socket. gitlab # [ 4.869972] systemd[1]: Starting Cleanup udev Database... gitlab # [ 4.871367] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. gitlab # [ 4.873509] systemd[1]: Stopped Create Static Device Nodes in /dev. gitlab # [ 4.875603] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. gitlab # [ 4.878079] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. gitlab # [ 4.881216] systemd[1]: kmod-static-nodes.service: Deactivated successfully. gitlab # [ 4.883394] systemd[1]: Stopped Create List of Static Device Nodes. gitlab # [ 4.896546] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. gitlab # [ 4.898987] systemd[1]: Finished Cleanup udev Database. gitlab # [ 4.903844] systemd[1]: Reached target Switch Root. gitlab # [ 4.906447] systemd[1]: Starting NixOS Activation... gitlab # [ 5.008571] initrd-nixos-activation-start[574]: booting system configuration /nix/store/0nnagcx9pqqdc7g7rq24ryyql9wypmy8-nixos-system-gitlab-test gitlab # [ 5.061746] initrd-nixos-activation-start[574]: running activation script... gitlab # [ 5.305638] initrd-nixos-activation-start[597]: setting up /etc... gitlab # [ 5.425811] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. gitlab # [ 5.434881] systemd[1]: Finished NixOS Activation. gitlab # [ 5.446860] systemd[1]: Starting Switch Root... gitlab # [ 5.487199] systemd[1]: Switching root. gitlab # [ 6.132379] systemd-journald[153]: Received SIGTERM from PID 1 (systemd). gitlab # [ 6.258697] NET: Registered PF_VSOCK protocol family gitlab # [ 6.636504] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 6.652461] systemd[1]: Detected virtualization kvm. gitlab # [ 6.655582] systemd[1]: Detected architecture x86-64. gitlab # [ 6.658816] systemd[1]: Detected first boot. gitlab # [ 6.662812] systemd[1]: Initializing machine ID from random generator. gitlab # [ 6.808340] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 6.879943] systemd[1]: Applying preset policy. gitlab # [ 7.040845] systemd[1]: Populated /etc with preset unit settings. gitlab # [ 7.161807] systemd[1]: initrd-switch-root.service: Deactivated successfully. gitlab # [ 7.164078] systemd[1]: Stopped initrd-switch-root.service. gitlab # [ 7.167641] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. gitlab # [ 7.170861] systemd[1]: Created slice Slice /system/getty. gitlab # [ 7.172996] systemd[1]: Created slice GitLab DevOps Platform Slice. gitlab # [ 7.175327] systemd[1]: Created slice User and Session Slice. gitlab # [ 7.176858] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 7.178938] systemd[1]: Started Forward Password Requests to Wall Directory Watch. gitlab # [ 7.180799] systemd[1]: Expecting device /dev/hvc0... gitlab # [ 7.182091] systemd[1]: Expecting device /dev/ttyS0... gitlab # [ 7.183422] systemd[1]: Reached target Local Encrypted Volumes. gitlab # [ 7.184832] systemd[1]: Stopped target initrd-fs.target. gitlab # [ 7.186249] systemd[1]: Stopped target initrd-root-fs.target. gitlab # [ 7.187636] systemd[1]: Stopped target initrd-switch-root.target. gitlab # [ 7.189146] systemd[1]: Reached target Virtual Machines and Containers. gitlab # [ 7.190796] systemd[1]: Reached target Path Units. gitlab # [ 7.192106] systemd[1]: Reached target Remote File Systems. gitlab # [ 7.193561] systemd[1]: Reached target Slice Units. gitlab # [ 7.194814] systemd[1]: Reached target Swaps. gitlab # [ 7.197079] systemd[1]: Listening on Process Core Dump Socket. gitlab # [ 7.199151] systemd[1]: Listening on Credential Encryption/Decryption. gitlab # [ 7.202517] systemd[1]: Starting Journal Log Access Socket... gitlab # [ 7.204700] systemd[1]: Listening on Journal Audit Socket. gitlab # [ 7.206402] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. gitlab # [ 7.208218] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 7.210554] systemd[1]: Listening on udev Control Socket. gitlab # [ 7.214574] systemd[1]: Mounting Huge Pages File System... gitlab # [ 7.217509] systemd[1]: Mounting POSIX Message Queue File System... gitlab # [ 7.220719] systemd[1]: Mounting Kernel Debug File System... gitlab # [ 7.225343] systemd[1]: Mounting Kernel Trace File System... gitlab # [ 7.233537] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 7.238370] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 7.250518] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 7.254402] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm gitlab # [ 7.260483] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 7.265887] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 7.273728] systemd[1]: Mounting FUSE Control File System... gitlab # [ 7.276860] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 7.298296] systemd[1]: Starting Journal Service... gitlab # [ 7.301462] systemd[1]: Starting Load Kernel Modules... gitlab # [ 7.306395] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... gitlab # [ 7.310389] systemd[1]: Starting Remount Root and Kernel File Systems... gitlab # [ 7.311992] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 7.317054] systemd-journald[791]: Collecting audit messages is enabled. gitlab # [ 7.318308] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 7.325286] systemd[1]: Listening on Journal Log Access Socket. gitlab # [ 7.327840] systemd[1]: Mounted Huge Pages File System. gitlab # [ 7.332417] systemd[1]: Mounted POSIX Message Queue File System. gitlab # [ 7.336688] systemd[1]: Mounted Kernel Debug File System. gitlab # [ 7.339757] systemd[1]: Mounted Kernel Trace File System. gitlab # [ 7.090873] systemd[1]: Queued start job for default target Multi-User System.[ 7.345626] systemd[1]: Finished Create List of Static Device Nodes. gitlab # gitlab # [ 7.095894] systemd[1]: Unnecessary job was r[ 7.349974] systemd[1]: Started Journal Service. gitlab # emoved for /dev/vdb. gitlab # [ 7.099881] systemd[1]: systemd-journald.service: Deactivated successfully. gitlab # [ 7.102343] systemd-modules-load[792]: Using 3 probe threads[ 7.357001] EXT4-fs (vda): re-mounted 3746b198-1d37-4ca2-b2b1-bc0868ba6636. gitlab # gitlab # [ 7.358972] loop: module loaded gitlab # [ 7.105623] systemd-modules-load[792]: Inserted module 'tls' gitlab # [ 7.107970] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 7.110240] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 7.112582] systemd[1]: Mounted FUSE Control File System. gitlab # [ 7.114472] systemd-modules-load[792]: Inserted module 'loop' gitlab # [ 7.116340] systemd[1]: Finished Load Kernel Modules. gitlab # [ 7.118451] systemd[1]: Finished Remount Root and Kernel File Systems. gitlab # [ 7.120469] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 7.124541] systemd[1]: Starting Firewall... gitlab # [ 7.126421] systemd[1]: Starting Flush Journal to Persistent Storage... gitlab # [ 7.130833] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 7.134108] systemd-oomd[794]: No swap; memory pressure usage will be degraded gitlab # [ 7.137183] systemd[1]: Starting Load/Save OS Random Seed... gitlab # [ 7.139911] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 7.397825] systemd-journald[791]: Received client request to flush runtime journal. gitlab # [ 7.216670] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 7.219482] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 7.222096] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. gitlab # [ 7.224223] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 7.226775] systemd[1]: Finished Load/Save OS Random Seed. gitlab # [ 7.229497] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 7.231253] systemd[1]: Reached target First Boot Complete. gitlab # [ 7.232922] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 7.234804] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 7.236561] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 7.238567] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 7.240441] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 7.242497] systemd[1]: Finished Flush Journal to Persistent Storage. gitlab # [ 7.244674] systemd-udevd[834]: Using default interface naming scheme 'v260'. gitlab # [ 7.282973] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 7.307811] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 7.411198] systemd[1]: Finished Firewall. gitlab # [ 7.413226] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 7.448823] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. gitlab # [ 7.450978] (udev-worker)[912]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 7.459985] (udev-worker)[921]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 7.463608] (udev-worker)[921]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 7.467533] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. gitlab # [ 7.478231] systemd[1]: Condition check resulted in Virtio network device being skipped. gitlab # [ 7.481641] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 7.484773] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 7.488775] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 7.491697] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 7.493969] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 7.788550] rtc_cmos 00:05: RTC can wake from S4 gitlab # [ 7.795960] rtc_cmos 00:05: registered as rtc0 gitlab # [ 7.797741] rtc_cmos 00:05: setting system clock to 2026-07-06T11:45:13 UTC (1783338313) gitlab # [ 7.800295] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs gitlab # [ 7.803078] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 gitlab # [ 7.806883] Floppy drive(s): fd0 is 2.88M AMI BIOS gitlab # [ 7.810931] parport_pc 00:03: reported by Plug and Play ACPI gitlab # [ 7.812570] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] gitlab # [ 7.820507] FDC 0 is a S82078B gitlab # [ 7.822819] mousedev: PS/2 mouse device common for all mice gitlab # [ 7.827685] ACPI: button: Power Button [PWRF] gitlab # [ 7.862366] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 gitlab # [ 7.865201] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console gitlab # [ 7.879134] Console: switching to colour dummy device 80x25 gitlab # [ 7.880420] [drm] Found bochs VGA, ID 0xb0c5. gitlab # [ 7.880422] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. gitlab # [ 7.880906] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic gitlab # [ 7.882996] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 gitlab # [ 7.884317] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0 gitlab # [ 7.896355] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input6 gitlab # [ 7.897972] fbcon: bochs-drmdrmfb (fb0) is primary device gitlab # [ 7.906138] Console: switching to colour frame buffer device 160x50 gitlab # [ 7.909005] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device gitlab # [ 7.915294] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 gitlab # [ 7.917249] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD gitlab # [ 7.724148] systemd[1]: Starting Virtual Console Setup... gitlab # [ 8.011146] ppdev: user-space parallel port driver gitlab # [ 7.766489] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 7.770164] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 7.786970] systemd[1]: Starting Virtual Console Setup... gitlab # [ 7.799099] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 7.801968] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 7.805373] systemd[1]: Starting Virtual Console Setup... gitlab # [ 8.085878] kvm_amd: TSC scaling supported gitlab # [ 8.087013] kvm_amd: Nested Virtualization enabled gitlab # [ 8.088149] kvm_amd: Nested Paging enabled gitlab # [ 8.089266] kvm_amd: LBR virtualization supported gitlab # [ 8.090385] kvm_amd: Virtual VMLOAD VMSAVE supported gitlab # [ 8.091599] kvm_amd: Virtual GIF supported gitlab # [ 8.092708] kvm_amd: Virtual NMI enabled gitlab # [ 7.892611] systemd-vconsole-setup[966]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 8.150624] EDAC MC: Ver: 3.0.0 gitlab # [ 7.896456] systemd[1]: Finished Virtual Console Setup. gitlab # [ 7.909452] systemd[1]: Mounting /run/wrappers... gitlab # [ 7.933139] systemd[1]: Mounted /run/wrappers. gitlab # [ 7.935441] systemd[1]: Reached target Local File Systems. gitlab # [ 7.937425] systemd[1]: Listening on Boot Loader Control Service Socket. gitlab # [ 7.940248] systemd[1]: Starting register-nix-paths.service... gitlab # [ 7.942857] systemd[1]: Starting Create SUID/SGID Wrappers... gitlab # [ 7.944511] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 7.959385] systemd[1]: Starting Save Transient machine-id to Disk... gitlab # [ 7.962069] systemd[1]: Starting Create System Files and Directories... gitlab # [ 8.001738] systemd[1]: Finished Create System Files and Directories. gitlab # [ 8.004584] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. gitlab # [ 8.007852] systemd[1]: Finished Save Transient machine-id to Disk. gitlab # [ 8.010895] systemd[1]: Starting Rebuild Journal Catalog... gitlab # [ 8.012635] systemd[1]: Starting Record System Boot/Shutdown in UTMP... gitlab # [ 8.040917] systemd[1]: Finished Record System Boot/Shutdown in UTMP. gitlab # [ 8.055534] systemd[1]: Finished Rebuild Journal Catalog. gitlab # [ 8.057961] systemd[1]: Starting Update is Completed... gitlab # [ 8.073236] systemd[1]: Finished Update is Completed. gitlab # [ 8.102667] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. gitlab # [ 8.104547] systemd[1]: Finished Create SUID/SGID Wrappers. gitlab # [ 8.105927] systemd[1]: suid-sgid-wrappers.service: Consumed 123ms CPU time over 159ms wall clock time, 4.4M memory peak, 1.1M read from disk. gitlab # [ 8.362836] systemd[1]: Finished register-nix-paths.service. gitlab # [ 8.366699] systemd[1]: Reached target System Initialization. gitlab # [ 8.370378] systemd[1]: Started Discard unused filesystem blocks once a week. gitlab # [ 8.374671] systemd[1]: Started logrotate.timer. gitlab # [ 8.377722] systemd[1]: Started Daily Cleanup of Temporary Directories. gitlab # [ 8.381617] systemd[1]: Reached target Timer Units. gitlab # [ 8.384818] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 8.388659] systemd[1]: Listening on Nix Daemon Socket. gitlab # [ 8.391939] systemd[1]: Starting nixbot CI service socket... gitlab # [ 8.395503] systemd[1]: Listening on Hostname Service Socket. gitlab # [ 8.399811] systemd[1]: Starting D-Bus System Message Bus... gitlab # [ 8.403393] systemd[1]: Listening on nixbot CI service socket. gitlab # [ 8.406984] systemd[1]: Reached target Socket Units. gitlab # [ 8.448130] dbus-broker-launch[1056]: Looking up NSS user entry for 'systemd-timesync'... gitlab # [ 8.453737] dbus-broker-launch[1056]: NSS returned no entry for 'systemd-timesync' gitlab # [ 8.458300] dbus-broker-launch[1056]: Invalid user-name in /nix/store/89zj9nmrajvcy9c3c08wv0ir9ww7lgmq-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" gitlab # [ 8.467714] systemd[1]: Started D-Bus System Message Bus. gitlab # [ 8.480645] dbus-broker-launch[1056]: Ready gitlab # [ 8.486609] systemd[1]: Reached target Basic System. gitlab # [ 8.490796] systemd[1]: Started backdoor.service. gitlab # [ 8.494611] systemd[1]: Starting gitlab-config.service... gitlab # [ 8.499714] systemd[1]: Starting Import lastlog data into lastlog2 database... gitlab # [ 8.512362] systemd[1]: Starting Logrotate configuration check... gitlab # [ 8.518750] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 8.526624] systemd[1]: Starting Post-Boot Actions... gitlab # [ 8.530381] logrotate[1061]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. gitlab # connecting to host... gitlab # [ 8.536679] logrotate[1061]: reading config file /nix/store/sn4bd2nqi3yg4r51djm9i2rmfgbwvmhb-logrotate.conf gitlab # [ 8.538508] logrotate[1061]: note: 'monthly' overrides previously specified 'weekly' gitlab # [ 8.539607] logrotate[1061]: Reading state from file: /var/lib/logrotate.status gitlab # [ 8.541894] logrotate[1061]: state file /var/lib/logrotate.status does not exist gitlab # [ 8.543428] logrotate[1061]: Allocating hash table for state file, size 64 entries gitlab: Guest shell says: b'Spawning backdoor root shell...\n' gitlab: connected to guest root shell gitlab # [ 8.547079] logrotate[1061]: Handling 4 logs gitlab: (connecting took 9.38 seconds) gitlab # [ 8.551216] logrotate[1061]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab: (finished: waiting for the VM to finish booting, in 9.38 seconds) gitlab # [ 8.557252] logrotate[1061]: considering log /var/log/btmp gitlab # [ 8.558803] logrotate[1061]: Creating new state gitlab # [ 8.561267] logrotate[1061]: Now: 2026-07-06 11:45 gitlab # [ 8.562492] logrotate[1061]: Last rotated at 2026-07-06 11:00 gitlab # [ 8.566240] logrotate[1061]: log does not need rotating (log has already been rotated) gitlab # [ 8.568207] logrotate[1061]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 8.571954] logrotate[1061]: considering log /var/log/wtmp gitlab # [ 8.573549] logrotate[1061]: Creating new state gitlab # [ 8.574706] logrotate[1061]: Now: 2026-07-06 11:45 gitlab # [ 8.576242] logrotate[1061]: Last rotated at 2026-07-06 11:00 gitlab # [ 8.577307] logrotate[1061]: log does not need rotating (log has already been rotated) gitlab # [ 8.578312] logrotate[1061]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed gitlab # [ 8.579090] logrotate[1061]: switching euid from 0 to 165 and egid from 0 to 165 (pid 1061) gitlab # [ 8.579929] logrotate[1061]: considering log /var/gitlab/state/log/*.log gitlab # [ 8.581463] logrotate[1061]: log /var/gitlab/state/log/*.log does not exist -- skipping gitlab # [ 8.582370] logrotate[1061]: Creating new state gitlab # [ 8.583194] logrotate[1061]: switching euid from 165 to 0 and egid from 165 to 0 (pid 1061) gitlab # [ 8.584767] logrotate[1061]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed gitlab # [ 8.585817] logrotate[1061]: switching euid from 0 to 60 and egid from 0 to 60 (pid 1061) gitlab # [ 8.586666] logrotate[1061]: considering log /var/log/nginx/*.log gitlab # [ 8.588269] logrotate[1061]: log /var/log/nginx/*.log does not exist -- skipping gitlab # [ 8.588957] logrotate[1061]: Creating new state gitlab # [ 8.590152] logrotate[1061]: switching euid from 60 to 0 and egid from 60 to 0 (pid 1061) gitlab # [ 8.591498] systemd[1]: Started Reset console on configuration changes. gitlab # [ 8.593304] systemd[1]: Starting resolvconf update... gitlab # [ 8.595157] systemd[1]: Finished Logrotate configuration check. gitlab # [ 8.597228] nsncd[1062]: Jul 06 11:45:14.552 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 8.598297] systemd[1]: Finished Post-Boot Actions. gitlab # [ 8.599787] systemd[1]: Finished Import lastlog data into lastlog2 database. gitlab # [ 8.601846] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 8.603959] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 8.605790] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 8.607132] systemd[1]: Starting User Login Management... gitlab # [ 8.636136] systemd[1]: Stopped target Host and Network Name Lookups. gitlab # [ 8.638215] systemd[1]: Stopping Host and Network Name Lookups... gitlab # [ 8.640627] systemd[1]: Stopped target User and Group Name Lookups. gitlab # [ 8.642056] systemd[1]: Stopping User and Group Name Lookups... gitlab # [ 8.646148] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... gitlab # [ 8.647255] systemd[1]: nscd.service: Deactivated successfully. gitlab # [ 8.649198] systemd[1]: Stopped Name Service Cache Daemon (nsncd). gitlab # [ 8.651610] systemd[1]: nscd.service: Consumed 15ms CPU time over 125ms wall clock time, 7.6M memory peak, 5.7M read from disk. gitlab # [ 8.655301] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 8.662928] systemd-logind[1107]: New seat seat0. gitlab # [ 8.666514] systemd-logind[1107]: Watching system buttons on /dev/input/event2 (Power Button) gitlab # [ 8.668139] systemd-logind[1107]: Watching system buttons on /dev/input/event5 (QEMU Virtio Keyboard) gitlab # [ 8.669251] systemd-logind[1107]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) gitlab # [ 8.670190] systemd[1]: Started User Login Management. gitlab # [ 8.672605] systemd[1]: Starting linger-users.service... gitlab # [ 8.690329] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 8.692329] nsncd[1149]: Jul 06 11:45:14.646 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 8.695931] systemd[1]: Finished resolvconf update. gitlab # [ 8.697476] systemd[1]: Reached target Preparation for Network. gitlab # [ 8.699315] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 8.700790] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 8.702860] systemd[1]: Starting DHCP Client... gitlab # [ 8.704371] systemd[1]: Starting Address configuration of eth1... gitlab # [ 8.705498] systemd[1]: Starting Extra networking commands.... gitlab # [ 8.707327] systemd[1]: linger-users.service: Deactivated successfully. gitlab # [ 8.712842] systemd[1]: Finished linger-users.service. gitlab # [ 8.733482] network-addresses-eth1-start[1175]: adding address 192.168.1.1/24... done gitlab # [ 8.736438] network-addresses-eth1-start[1175]: adding address 2001:db8:1::1/64... done gitlab # [ 8.742344] systemd[1]: Finished Address configuration of eth1. gitlab # [ 8.782138] systemd[1]: Finished Extra networking commands.. gitlab # [ 8.783760] systemd[1]: Reached target Network. gitlab # [ 8.786325] systemd[1]: Starting gitlab-workhorse.service... gitlab # [ 8.789479] systemd[1]: Starting Nginx Web Server... gitlab # [ 8.792795] dhcpcd[1193]: dhcpcd-10.3.2 starting gitlab # [ 8.794103] systemd[1]: Starting PostgreSQL Server... gitlab # [ 8.798609] systemd[1]: Starting Redis Server - redis-gitlab... gitlab # [ 8.800320] dhcpcd[1237]: dev: loaded udev gitlab # [ 9.060328] 8021q: 802.1Q VLAN Support v1.8 gitlab # [ 9.061139] 8021q: adding VLAN 0 to HW filter on device eth1 gitlab # [ 8.807545] systemd[1]: Starting Permit User Sessions... gitlab # [ 8.824802] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. gitlab # [ 8.829971] systemd[1]: Finished Permit User Sessions. gitlab # [ 8.840099] systemd[1]: Started Getty on tty1. gitlab # [ 8.841233] systemd[1]: Reached target Login Prompts. gitlab # [ 9.173649] cfg80211: Loading compiled-in X.509 certificates for regulatory database gitlab # [ 9.184835] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' gitlab # [ 9.187398] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' gitlab # [ 9.190161] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 gitlab # [ 9.191803] cfg80211: failed to load regulatory.db gitlab # [ 8.949961] nginx-pre-start[1257]: nginx: the configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf syntax is ok gitlab # [ 8.953100] nginx-pre-start[1257]: nginx: configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf test is successful gitlab # [ 8.960527] systemd[1]: Started Nginx Web Server. gitlab # [ 8.963266] postgresql-pre-start[1259]: The files belonging to this database system will be owned by user "postgres". gitlab # [ 8.964456] postgresql-pre-start[1259]: This user must also own the server process. gitlab # [ 8.970869] postgresql-pre-start[1259]: The database cluster will be initialized with locale "en_US.UTF-8". gitlab # [ 8.971982] postgresql-pre-start[1259]: The default database encoding has accordingly been set to "UTF8". gitlab # [ 8.975131] postgresql-pre-start[1259]: The default text search configuration will be set to "english". gitlab # [ 8.976165] postgresql-pre-start[1259]: Data page checksums are disabled. gitlab # [ 8.977121] postgresql-pre-start[1259]: fixing permissions on existing directory /var/lib/postgresql/17 ... ok gitlab # [ 8.977815] postgresql-pre-start[1259]: creating subdirectories ... ok gitlab # [ 8.981633] postgresql-pre-start[1259]: selecting dyn[ 9.237331] 8021q: adding VLAN 0 to HW filter on device eth0 gitlab # amic shared memory implementation ... posix gitlab # [ 8.985279] dhcpcd[1237]: eth0: waiting for carrier gitlab # [ 8.986047] dhcpcd[1237]: eth0: carrier acquired gitlab # [ 8.991776] redis-server[1261]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. gitlab # [ 8.994243] redis-server[1261]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo gitlab # [ 8.996258] redis-server[1261]: Redis version=8.8.0, bits=64, commit=00000000, modified=0, pid=1261, just started gitlab # [ 8.997287] systemd[1]: Started Redis Server - redis-gitlab. gitlab # [ 8.998213] redis-server[1261]: Configuration loaded gitlab # [ 8.999054] redis[1261]: monotonic clock: POSIX clock_gettime gitlab # [ 8.999926] dhcpcd[1237]: DUID 00:01:00:01:31:de:51:ca:52:54:00:12:34:56 gitlab # [ 9.001201] dhcpcd[1237]: eth0: IAID 00:12:34:56 gitlab # [ 9.002175] dhcpcd[1237]: eth0: adding address fe80::5054:ff:fe12:3456 gitlab # [ 9.003139] redis[1261]: Running mode=standalone, port=0. gitlab # [ 9.004210] redis[1261]: Server initialized gitlab # [ 9.005196] redis[1261]: Ready to accept connections unix gitlab # [ 9.005781] redis[1261]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. gitlab # [ 9.009955] postgresql-pre-start[1259]: selecting default "max_connections" ... 100 gitlab # [ 9.051686] postgresql-pre-start[1259]: selecting default "shared_buffers" ... 128MB gitlab # [ 9.269129] systemd[1]: Started gitlab-workhorse.service. gitlab # [ 9.471116] postgresql-pre-start[1259]: selecting default time zone ... UTC gitlab # [ 9.474711] postgresql-pre-start[1259]: creating configuration files ... ok gitlab # [ 9.556741] gitlab-workhorse[1293]: time="2026-07-06T11:45:15Z" level=info msg=Starting build_time=19700101.000000 version=18.11.4 gitlab # [ 9.559151] gitlab-workhorse[1293]: time="2026-07-06T11:45:15Z" level=info msg="Using redis/go-redis" gitlab # [ 9.560778] gitlab-workhorse[1293]: time="2026-07-06T11:45:15Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix gitlab # [ 9.675338] postgresql-pre-start[1259]: running bootstrap script ... ok gitlab # [ 10.147653] postgresql-pre-start[1259]: performing post-bootstrap initialization ... ok gitlab # [ 10.357503] dhcpcd[1237]: eth0: soliciting a DHCP lease gitlab # [ 10.623844] NET: Registered PF_PACKET protocol family gitlab # [ 10.383969] dhcpcd[1237]: eth0: offered 10.0.2.15 from 10.0.2.2 gitlab # [ 10.388879] dhcpcd[1237]: eth0: probing address 10.0.2.15/24 gitlab # [ 10.551624] postgresql-pre-start[1259]: syncing data to disk ... ok gitlab # [ 10.553123] postgresql-pre-start[1259]: initdb: warning: enabling "trust" authentication for local connections gitlab # [ 10.555691] postgresql-pre-start[1259]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. gitlab # [ 10.557824] postgresql-pre-start[1259]: Success. You can now start the database server using: gitlab # [ 10.561318] postgresql-pre-start[1259]: pg_ctl -D /var/lib/postgresql/17 -l logfile start gitlab # [ 10.689181] postgres[1320]: [1320] LOG: starting PostgreSQL 17.10 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit gitlab # [ 10.691190] postgres[1320]: [1320] LOG: listening on IPv6 address "::1", port 5432 gitlab # [ 10.692540] postgres[1320]: [1320] LOG: listening on IPv4 address "127.0.0.1", port 5432 gitlab # [ 10.697139] postgres[1320]: [1320] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" gitlab # [ 10.716364] postgres[1331]: [1331] LOG: database system was shut down at 2026-07-06 11:45:16 GMT gitlab # [ 10.727191] postgres[1320]: [1320] LOG: database system is ready to accept connections gitlab # [ 10.734542] systemd[1]: Started PostgreSQL Server. gitlab # [ 10.739407] systemd[1]: Starting PostgreSQL Setup Scripts... gitlab # [ 10.896678] postgresql-setup-start[1347]: CREATE DATABASE gitlab # [ 10.933368] postgresql-setup-start[1356]: CREATE ROLE gitlab # [ 10.956761] postgresql-setup-start[1358]: ALTER DATABASE gitlab # [ 10.995827] postgresql-setup-start[1364]: CREATE ROLE gitlab # [ 11.000281] systemd[1]: Finished PostgreSQL Setup Scripts. gitlab # [ 11.003458] systemd[1]: Reached target PostgreSQL. gitlab # [ 11.005860] systemd[1]: Starting gitlab-postgresql.service... gitlab # [ 11.067100] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1355]: mkdir -p /var/gitlab/state/home/.ssh: OK gitlab # [ 11.074465] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1355]: chmod 700 /var/gitlab/state/home/.ssh: OK gitlab # [ 11.113361] gitlab-postgresql-start[1372]: CREATE DATABASE gitlab # [ 11.124676] systemd[1]: Finished gitlab-config.service. gitlab # [ 11.129614] systemd[1]: Started gitaly.service. gitlab # [ 11.133971] systemd[1]: Starting gitlab-db-config.service... gitlab # [ 11.152827] gitlab-postgresql-start[1388]: CREATE EXTENSION gitlab # [ 11.181594] dhcpcd[1237]: eth0: soliciting an IPv6 router gitlab # [ 11.186492] dhcpcd[1237]: eth0: Router Advertisement from fe80::2 gitlab # [ 11.190566] dhcpcd[1237]: eth0: adding address fec0::5054:ff:fe12:3456/64 gitlab # [ 11.194976] dhcpcd[1237]: eth0: adding route to fec0::/64 gitlab # [ 11.199187] dhcpcd[1237]: eth0: adding default route via fe80::2 gitlab # [ 11.202445] gitlab-postgresql-start[1394]: CREATE EXTENSION gitlab # [ 11.205388] systemd[1]: Finished gitlab-postgresql.service. gitlab # [ 11.743472] gitaly[1389]: time="2026-07-06T11:45:17.698Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/2wcq0nlg508fs5wq7jnpg1lw2pib950g-gitaly.toml' and fix the errors that are printed." pid=1389 gitlab # [ 11.745541] gitaly[1389]: time="2026-07-06T11:45:17.699Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=1389 gitlab # [ 11.747341] gitaly[1389]: time="2026-07-06T11:45:17.699Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=1389 gitlab # [ 11.749176] gitaly[1389]: time="2026-07-06T11:45:17.699Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=1389 gitlab # [ 11.750234] gitaly[1389]: time="2026-07-06T11:45:17.699Z" level=info msg="Starting Gitaly" pid=1389 version=18.11.4 gitlab # [ 11.972170] gitaly[1389]: time="2026-07-06T11:45:17.928Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=208 pid=1389 gitlab # [ 11.974056] gitaly[1389]: time="2026-07-06T11:45:17.928Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=1389 gitlab # [ 11.978326] gitaly[1389]: time="2026-07-06T11:45:17.934Z" level=info msg="finished initializing command factory" duration_ms=6 pid=1389 gitlab # [ 11.979754] gitaly[1389]: time="2026-07-06T11:45:17.934Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-1389/git-exec-204586547.d/git pid=1389 gitlab # [ 11.984621] gitaly[1389]: time="2026-07-06T11:45:17.940Z" level=info msg="finished detecting git version" duration_ms=6 pid=1389 gitlab # [ 11.985692] gitaly[1389]: time="2026-07-06T11:45:17.941Z" level=info msg="using Git version" pid=1389 version=2.53.g gitlab # [ 11.986996] gitaly[1389]: time="2026-07-06T11:45:17.941Z" level=info msg="clearing disk cache object folder" pid=1389 storage=default gitlab # [ 11.987976] gitaly[1389]: time="2026-07-06T11:45:17.942Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2355391729 pid=1389 storage=default gitlab # [ 11.989347] gitaly[1389]: time="2026-07-06T11:45:17.942Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2355391729 pid=1389 storage=default gitlab # [ 11.990520] gitaly[1389]: time="2026-07-06T11:45:17.942Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=1389 gitlab # [ 11.991428] gitaly[1389]: time="2026-07-06T11:45:17.942Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=1389 gitlab # [ 11.992425] gitaly[1389]: time="2026-07-06T11:45:17.943Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2355391729 clear_duration_ms=0 pid=1389 storage=default gitlab # [ 12.088290] gitaly[1389]: time="2026-07-06T11:45:18.044Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=1389 gitlab # [ 12.090148] gitaly[1389]: time="2026-07-06T11:45:18.044Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=1389 schema=unix gitlab # [ 12.091671] gitaly[1389]: time="2026-07-06T11:45:18.044Z" level=info msg="counting repositories" pid=1389 gitlab # [ 12.093894] gitaly[1389]: time="2026-07-06T11:45:18.044Z" level=info msg="starting to count repositories" pid=1389 storage_path=/var/gitlab/state/repositories gitlab # [ 12.096400] gitaly[1389]: time="2026-07-06T11:45:18.044Z" level=info msg="finished tempdir cleaner walk" pid=1389 storage=default time_ms=0 gitlab # [ 12.097646] gitaly[1389]: time="2026-07-06T11:45:18.045Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=1389 storage_path=/var/gitlab/state/repositories gitlab # [ 12.099490] gitaly[1389]: time="2026-07-06T11:45:18.045Z" level=info msg="completed counting all repositories" counting_duration=0 pid=1389 gitlab # [ 12.100722] gitaly[1389]: time="2026-07-06T11:45:18.044Z" level=info msg="listening at address" address=/run/gitaly/gitaly-1389/sock.d/intern pid=1389 schema=unix gitlab # [ 12.124711] gitaly[1389]: time="2026-07-06T11:45:18.081Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=1389 gitlab # [ 12.125765] gitaly[1389]: time="2026-07-06T11:45:18.081Z" level=info msg="Started Gitaly" duration_ms=381 pid=1389 gitlab # [ 12.130447] gitaly[1389]: time="2026-07-06T11:45:18.086Z" level=info msg="maintenance: daily scheduled" pid=1389 scheduled="2026-07-06 12:00:00 +0000 UTC" gitlab # [ 12.514332] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1393]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 12.516582] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1393]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 12.561480] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1393]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 14.120451] gitaly[1389]: time="2026-07-06T11:45:20.075Z" level=info msg="License database preloaded" duration_ms=2375 pid=1389 gitlab # [ 15.015979] dhcpcd[1237]: eth0: leased 10.0.2.15 for 86400 seconds gitlab # [ 15.020499] dhcpcd[1237]: eth0: adding route to 10.0.2.0/24 gitlab # [ 15.024186] dhcpcd[1237]: eth0: adding default route via 10.0.2.2 gitlab # [ 15.152972] systemd[1]: Started DHCP Client. gitlab # [ 15.155979] systemd[1]: Reached target Network is Online. gitlab # [ 45.054626] postgres[1767]: [1767] ERROR: relation "feature_gates" does not exist at character 523 gitlab # [ 45.056992] postgres[1767]: [1767] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), gitlab # [ 45.059132] postgres[1767]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, gitlab # [ 45.061403] postgres[1767]: c.collname, col_description(a.attrelid, a.attnum) AS comment, gitlab # [ 45.063525] postgres[1767]: attidentity AS identity, gitlab # [ 45.064928] postgres[1767]: attgenerated as attgenerated gitlab # [ 45.066476] postgres[1767]: FROM pg_attribute a gitlab # [ 45.067691] postgres[1767]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum gitlab # [ 45.069841] postgres[1767]: LEFT JOIN pg_type t ON a.atttypid = t.oid gitlab # [ 45.071494] postgres[1767]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation gitlab # [ 45.073862] postgres[1767]: WHERE a.attrelid = '"feature_gates"'::regclass gitlab # [ 45.075556] postgres[1767]: AND a.attnum > 0 AND NOT a.attisdropped gitlab # [ 45.077150] postgres[1767]: ORDER BY a.attnum gitlab # [ 45.078357] postgres[1767]: gitlab # [ 45.299464] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1393]: W, [2026-07-06T11:45:51.254737 #1393] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 49.625244] gitlab-workhorse[1293]: time="2026-07-06T11:45:55Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 50.793170] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1828]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping gitlab # [ 50.797396] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1828]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping gitlab # [ 59.635206] gitlab-workhorse[1293]: time="2026-07-06T11:46:05Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 68.999445] postgres[1912]: [1912] ERROR: relation "table_without_models" does not exist gitlab # [ 69.002684] postgres[1912]: [1912] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 69.645083] gitlab-workhorse[1293]: time="2026-07-06T11:46:15Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 79.656530] gitlab-workhorse[1293]: time="2026-07-06T11:46:25Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 85.348327] systemd[1]: Finished gitlab-db-config.service. gitlab # [ 85.358220] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 85.364511] systemd[1]: Starting gitlab.service... gitlab # [ 85.764522] bundle[2133]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 85.768305] bundle[2133]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 85.771571] bundle[2133]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 88.880305] sidekiq-cluster[2132]: {"severity":"INFO","time":"2026-07-06T11:46:34.817Z","message":"Starting cluster with 1 processes"} gitlab # [ 88.934249] bundle[2133]: {"timestamp":"2026-07-06T11:46:34.867Z","pid":2133,"message":"* Enabling systemd notification integration"} gitlab # [ 88.935794] bundle[2133]: {"timestamp":"2026-07-06T11:46:34.890Z","pid":2133,"message":"Puma starting in cluster mode..."} gitlab # [ 88.936795] bundle[2133]: {"timestamp":"2026-07-06T11:46:34.890Z","pid":2133,"message":"* Puma version: 7.2.0 (\"On The Corner\")"} gitlab # [ 88.937754] bundle[2133]: {"timestamp":"2026-07-06T11:46:34.890Z","pid":2133,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 88.938797] bundle[2133]: {"timestamp":"2026-07-06T11:46:34.890Z","pid":2133,"message":"* Min threads: 0"} gitlab # [ 88.939985] bundle[2133]: {"timestamp":"2026-07-06T11:46:34.890Z","pid":2133,"message":"* Max threads: 4"} gitlab # [ 88.941149] bundle[2133]: {"timestamp":"2026-07-06T11:46:34.890Z","pid":2133,"message":"* Environment: production"} gitlab # [ 88.941974] bundle[2133]: {"timestamp":"2026-07-06T11:46:34.890Z","pid":2133,"message":"* Master PID: 2133"} gitlab # [ 88.943144] bundle[2133]: {"timestamp":"2026-07-06T11:46:34.890Z","pid":2133,"message":"* Workers: 2"} gitlab # [ 88.943961] bundle[2133]: {"timestamp":"2026-07-06T11:46:34.890Z","pid":2133,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} gitlab # [ 88.945173] bundle[2133]: {"timestamp":"2026-07-06T11:46:34.891Z","pid":2133,"message":"* Preloading application"} gitlab # [ 89.666452] gitlab-workhorse[1293]: time="2026-07-06T11:46:35Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 99.668504] gitlab-workhorse[1293]: time="2026-07-06T11:46:45Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 109.670204] gitlab-workhorse[1293]: time="2026-07-06T11:46:55Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 114.189729] bundle[2133]: W, [2026-07-06T11:47:00.142975 #2133] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 118.283141] sidekiq-cluster[2163]: 2026-07-06T11:47:04.237Z pid=2163 tid=u0r INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2163, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>430174208} gitlab # [ 118.485152] sidekiq-cluster[2163]: W, [2026-07-06T11:47:04.440530 #2163] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 118.531713] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:04.487Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 119.679679] gitlab-workhorse[1293]: time="2026-07-06T11:47:05Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 120.678571] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.633Z","message":"GitLab reliable fetch activated!"} gitlab # [ 120.680809] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.634Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 120.709573] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.665Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} gitlab # [ 120.714131] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.670Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} gitlab # [ 120.722854] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.679Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} gitlab # [ 120.729341] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.685Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} gitlab # [ 120.733263] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.689Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} gitlab # [ 120.737128] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.693Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} gitlab # [ 120.740759] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.696Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} gitlab # [ 120.744237] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.700Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} gitlab # [ 120.747758] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.703Z","message":"Cron Jobs - added job with name expire_build_artifacts_worker in the namespace default"} gitlab # [ 120.751381] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.707Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} gitlab # [ 120.754923] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.711Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} gitlab # [ 120.759233] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.715Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} gitlab # [ 120.762684] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.718Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} gitlab # [ 120.766251] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.722Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} gitlab # [ 120.769850] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.726Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} gitlab # [ 120.773264] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.729Z","message":"Cron Jobs - added job with name environments_auto_delete_cron_worker in the namespace default"} gitlab # [ 120.776594] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.732Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} gitlab # [ 120.780480] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.736Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} gitlab # [ 120.784133] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.740Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} gitlab # [ 120.787523] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.743Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} gitlab # [ 120.790976] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.747Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} gitlab # [ 120.794620] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.750Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} gitlab # [ 120.797858] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.754Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} gitlab # [ 120.801147] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.757Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} gitlab # [ 120.804243] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.760Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} gitlab # [ 120.807238] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.763Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} gitlab # [ 120.810833] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.767Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} gitlab # [ 120.813487] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.769Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} gitlab # [ 120.816118] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.772Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} gitlab # [ 120.818859] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.775Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} gitlab # [ 120.821706] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.777Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} gitlab # [ 120.824177] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.780Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} gitlab # [ 120.827151] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.783Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} gitlab # [ 120.829809] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.786Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} gitlab # [ 120.832687] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.788Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} gitlab # [ 120.835273] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.791Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} gitlab # [ 120.837721] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.793Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} gitlab # [ 120.840231] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.796Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} gitlab # [ 120.842781] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.799Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} gitlab # [ 120.845130] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.801Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} gitlab # [ 120.847323] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.803Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} gitlab # [ 120.849910] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.806Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} gitlab # [ 120.852269] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.808Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} gitlab # [ 120.854814] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.811Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} gitlab # [ 120.857420] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.813Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} gitlab # [ 120.860761] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.817Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} gitlab # [ 120.863515] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.819Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} gitlab # [ 120.865816] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.822Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} gitlab # [ 120.868426] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.824Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} gitlab # [ 120.871208] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.827Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} gitlab # [ 120.873778] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.830Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} gitlab # [ 120.876583] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.832Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} gitlab # [ 120.879536] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.835Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} gitlab # [ 120.889138] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.845Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} gitlab # [ 120.892145] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.848Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} gitlab # [ 120.894850] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.851Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} gitlab # [ 120.897446] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.853Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} gitlab # [ 120.900088] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.856Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} gitlab # [ 120.902953] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.859Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} gitlab # [ 120.905410] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.861Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} gitlab # [ 120.908156] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.864Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} gitlab # [ 120.910779] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.867Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} gitlab # [ 120.913160] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.869Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} gitlab # [ 120.916217] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.872Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} gitlab # [ 120.919227] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.875Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} gitlab # [ 120.922154] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.878Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} gitlab # [ 120.947182] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.902Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} gitlab # [ 120.949700] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.905Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} gitlab # [ 120.951864] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.908Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} gitlab # [ 120.954249] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.910Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} gitlab # [ 120.956592] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.912Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} gitlab # [ 120.959794] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.916Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} gitlab # [ 120.962220] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.918Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} gitlab # [ 120.965260] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.921Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} gitlab # [ 120.967597] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.923Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} gitlab # [ 120.970347] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.926Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} gitlab # [ 120.972617] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.928Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} gitlab # [ 120.975563] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.931Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} gitlab # [ 120.977923] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.934Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} gitlab # [ 120.980236] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.936Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} gitlab # [ 120.983136] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.939Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} gitlab # [ 120.985227] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.941Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} gitlab # [ 120.987961] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.944Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} gitlab # [ 120.990635] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.946Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} gitlab # [ 120.993415] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.949Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} gitlab # [ 120.995652] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.951Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} gitlab # [ 120.998646] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.954Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} gitlab # [ 121.005293] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.961Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} gitlab # [ 121.011073] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.967Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} gitlab # [ 121.014757] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.971Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} gitlab # [ 121.017175] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.973Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} gitlab # [ 121.020239] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.976Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} gitlab # [ 121.022534] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.978Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} gitlab # [ 121.025551] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.981Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} gitlab # [ 121.027973] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.984Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} gitlab # [ 121.031072] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.987Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} gitlab # [ 121.033888] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.990Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} gitlab # [ 121.036604] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.992Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} gitlab # [ 121.038869] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:06.995Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} gitlab # [ 121.049506] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:07.005Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} gitlab # [ 121.051796] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:07.008Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} gitlab # [ 121.054090] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:07.010Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} gitlab # [ 121.057700] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:07.014Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} gitlab # [ 121.061094] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:07.017Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} gitlab # [ 121.063826] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:07.020Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} gitlab # [ 121.066825] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:07.023Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} gitlab # [ 121.069084] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:07.025Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} gitlab # [ 121.071756] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:07.028Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} gitlab # [ 121.074734] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:07.031Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} gitlab # [ 121.077346] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:07.033Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} gitlab # [ 129.682983] gitlab-workhorse[1293]: time="2026-07-06T11:47:15Z" level=error correlation_id= duration_ms=1 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 139.685567] gitlab-workhorse[1293]: time="2026-07-06T11:47:25Z" level=error correlation_id= duration_ms=1 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 143.886848] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:29.841Z","message":"Booted Rails 7.2.3 application in production environment"} gitlab # [ 143.891117] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:29.843Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 143.894313] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:29.843Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 143.896966] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:29.843Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 143.918220] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:29.874Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 145.366861] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:31.322Z","message":"Cleaning working queues"} gitlab # [ 145.371340] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:31.324Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"a4015f7431c4c01da13e2159","created_at":"2026-07-06T11:46:30.635Z","trace_propagation_headers":{"sentry-trace":"56d3fe5fab5c46d2b12f60e27296488e-e02559ca6fc2454e","baggage":"sentry-trace_id=56d3fe5fab5c46d2b12f60e27296488e,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"7eee9d36aad57a672b56649a6a7db749","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-06T11:46:30.647Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-a4015f7431c4c01da13e2159: start","job_status":"start","queue_duration_s":60.676359,"scheduling_latency_s":60.67642} gitlab # [ 145.516390] bundle[2133]: {"timestamp":"2026-07-06T11:47:31.471Z","pid":2133,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} gitlab # [ 145.524799] bundle[2133]: {"timestamp":"2026-07-06T11:47:31.471Z","pid":2133,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} gitlab # [ 145.531770] bundle[2133]: {"timestamp":"2026-07-06T11:47:31.471Z","pid":2133,"message":"! #\u003cThread:0x00007ddb6f9ce7d0 /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} gitlab # [ 145.552486] bundle[2133]: {"timestamp":"2026-07-06T11:47:31.471Z","pid":2133,"message":"! #\u003cThread:0x00007ddb6fbef028 /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} gitlab # [ 145.569365] bundle[2133]: {"timestamp":"2026-07-06T11:47:31.472Z","pid":2133,"message":"Use Ctrl-C to stop"} gitlab # [ 145.573511] bundle[2133]: {"timestamp":"2026-07-06T11:47:31.472Z","pid":2133,"message":"Sending status to systemd every 1.0 sec"} gitlab # [ 145.576715] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:47:31.519Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"a4015f7431c4c01da13e2159","created_at":"2026-07-06T11:46:30.635Z","trace_propagation_headers":{"sentry-trace":"56d3fe5fab5c46d2b12f60e27296488e-e02559ca6fc2454e","baggage":"sentry-trace_id=56d3fe5fab5c46d2b12f60e27296488e,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"7eee9d36aad57a672b56649a6a7db749","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-06T11:46:30.647Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-a4015f7431c4c01da13e2159: done: 0.195315 sec","job_status":"done","queue_duration_s":60.676359,"scheduling_latency_s":60.67642,"redis_calls":26,"redis_duration_s":0.007075,"redis_read_bytes":914,"redis_write_bytes":2887,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.003743,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001852,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00148,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.073078,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.083222309,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.195315,"completed_at":"2026-07-06T11:47:31.519Z","load_balancing_strategy":"primary","db_duration_s":0.051554,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 145.881136] bundle[2133]: {"timestamp":"2026-07-06T11:47:31.835Z","pid":2133,"message":"- Worker 1 (PID: 2713) booted in 0.31s, phase: 0"} gitlab # [ 145.913884] bundle[2133]: {"timestamp":"2026-07-06T11:47:31.869Z","pid":2133,"message":"- Worker 0 (PID: 2709) booted in 0.36s, phase: 0"} gitlab # [ 145.921577] systemd[1]: Started gitlab.service. gitlab # [ 145.924956] systemd[1]: Reached target Common target for all GitLab services.. gitlab # [ 145.929534] systemd[1]: Starting setup-gitlab.service... gitlab # [ 145.953230] setup-gitlab-start[2737]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' gitlab # [ 148.483637] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:47:34 +0000] "GET /users/sign_in HTTP/1.1" 200 10624 "" "curl/8.20.0" 2503 gitlab # [ 148.497588] setup-gitlab-start[2761]: ++ curl -fs -X POST -H 'Content-Type: application/json' -d '{"grant_type":"password","username":"root","password":"notproduction"}' http://gitlab/oauth/token gitlab # [ 148.500398] setup-gitlab-start[2762]: ++ jq -r .access_token gitlab # [ 149.291482] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:47:35 +0000] "POST /oauth/token HTTP/1.1" 200 244 "" "curl/8.20.0" 768 gitlab # [ 149.303567] setup-gitlab-start[2737]: + BEARER=d671e376603561162e9671344be06509817bf784d0b1056733bc6ad47e430cfa gitlab # [ 149.306122] setup-gitlab-start[2737]: + AUTH='Authorization: Bearer d671e376603561162e9671344be06509817bf784d0b1056733bc6ad47e430cfa' gitlab # [ 149.308933] setup-gitlab-start[2737]: + curl -fs -X PUT -H 'Authorization: Bearer d671e376603561162e9671344be06509817bf784d0b1056733bc6ad47e430cfa' 'http://gitlab/api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true' gitlab # [ 150.096580] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:47:36 +0000] "PUT /api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true HTTP/1.1" 200 17133 "" "curl/8.20.0" 767 gitlab # [ 150.105970] setup-gitlab-start[2737]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner Rails.cache.clear gitlab # [ 150.128483] runuser[2779]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) gitlab # [ 150.508620] setup-gitlab-start[2780]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 150.511992] setup-gitlab-start[2780]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 150.514113] setup-gitlab-start[2780]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 151.105982] setup-gitlab-start[2780]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 151.110196] setup-gitlab-start[2780]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 151.112775] setup-gitlab-start[2780]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 178.315372] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:04.270Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"b33534ab0d3eb5694a58bee0","created_at":"2026-07-06T11:48:04.190Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f9583666eb4e2cbcc93b5f3def4a395f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.268Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-b33534ab0d3eb5694a58bee0: start","job_status":"start","queue_duration_s":0.002181,"scheduling_latency_s":0.002219} gitlab # [ 178.399559] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:04.354Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"b33534ab0d3eb5694a58bee0","created_at":"2026-07-06T11:48:04.190Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f9583666eb4e2cbcc93b5f3def4a395f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.268Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-b33534ab0d3eb5694a58bee0: done: 0.084494 sec","job_status":"done","queue_duration_s":0.002181,"scheduling_latency_s":0.002219,"redis_calls":13,"redis_duration_s":0.015437,"redis_read_bytes":139,"redis_write_bytes":1636,"redis_feature_flag_calls":5,"redis_feature_flag_duration_s":0.006582,"redis_feature_flag_read_bytes":132,"redis_feature_flag_write_bytes":698,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006749,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002106,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018887,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000953473,"duration_s":0.084494,"completed_at":"2026-07-06T11:48:04.354Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020008,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 178.406570] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:04.362Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"dc4ccee5d112b0adf65f0aac","created_at":"2026-07-06T11:48:04.278Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"70cae59c3e5e6098ffe34411c6ee89b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.290Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-dc4ccee5d112b0adf65f0aac: start","job_status":"start","queue_duration_s":0.071514,"scheduling_latency_s":0.071569} gitlab # [ 178.465722] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:04.421Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"dc4ccee5d112b0adf65f0aac","created_at":"2026-07-06T11:48:04.278Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"70cae59c3e5e6098ffe34411c6ee89b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.290Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-dc4ccee5d112b0adf65f0aac: done: 0.059466 sec","job_status":"done","queue_duration_s":0.071514,"scheduling_latency_s":0.071569,"redis_calls":19,"redis_duration_s":0.01226,"redis_read_bytes":21,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002391,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000722,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.007169,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001978,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028738,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000546717,"duration_s":0.059466,"completed_at":"2026-07-06T11:48:04.421Z","load_balancing_strategy":"primary","db_duration_s":0.007867,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 178.487235] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:04.430Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"eaf86ffa7132a82eb26daa53","created_at":"2026-07-06T11:48:04.303Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dd6b56febf1fc79c286aac6795d1417b","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.315Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-eaf86ffa7132a82eb26daa53: start","job_status":"start","queue_duration_s":0.114307,"scheduling_latency_s":0.114334} gitlab # [ 178.551368] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:04.507Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"eaf86ffa7132a82eb26daa53","created_at":"2026-07-06T11:48:04.303Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dd6b56febf1fc79c286aac6795d1417b","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.315Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-eaf86ffa7132a82eb26daa53: done: 0.077355 sec","job_status":"done","queue_duration_s":0.114307,"scheduling_latency_s":0.114334,"redis_calls":16,"redis_duration_s":0.00675,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.00331,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002552,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000888,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021983,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00104287,"duration_s":0.077355,"completed_at":"2026-07-06T11:48:04.507Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.039854,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 178.575868] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:04.517Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"dbc831338c7f952f63972920","created_at":"2026-07-06T11:48:04.327Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2271da969833dca201fda4615437b7e5","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.341Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-dbc831338c7f952f63972920: start","job_status":"start","queue_duration_s":0.175735,"scheduling_latency_s":0.175781} gitlab # [ 179.192569] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:05.148Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"dbc831338c7f952f63972920","created_at":"2026-07-06T11:48:04.327Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2271da969833dca201fda4615437b7e5","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.341Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-dbc831338c7f952f63972920: done: 0.630947 sec","job_status":"done","queue_duration_s":0.175735,"scheduling_latency_s":0.175781,"redis_calls":574,"redis_duration_s":0.310004,"redis_read_bytes":573,"redis_write_bytes":49866,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00121,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002198,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.306596,"redis_shared_state_read_bytes":567,"redis_shared_state_write_bytes":49017,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.355252,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00042659,"duration_s":0.630947,"completed_at":"2026-07-06T11:48:05.148Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002668,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 179.210634] setup-gitlab-start[2780]: W, [2026-07-06T11:48:05.166224 #2780] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 179.262125] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:05.217Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"ad81cc59c87c65013846de57","created_at":"2026-07-06T11:48:04.359Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"baedca7fcfb201b6e4d0ee9d65e458ac","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.368Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-ad81cc59c87c65013846de57: start","job_status":"start","queue_duration_s":0.848881,"scheduling_latency_s":0.848924} gitlab # [ 179.304622] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:05.260Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 179.325523] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:05.280Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"ad81cc59c87c65013846de57","created_at":"2026-07-06T11:48:04.359Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"baedca7fcfb201b6e4d0ee9d65e458ac","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.368Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-ad81cc59c87c65013846de57: done: 0.062945 sec","job_status":"done","queue_duration_s":0.848881,"scheduling_latency_s":0.848924,"redis_calls":10,"redis_duration_s":0.008264,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00104,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006523,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000701,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022732,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000740597,"duration_s":0.062945,"completed_at":"2026-07-06T11:48:05.280Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001142,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 179.392359] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:05.348Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"90e373fbefce7ff9229c3e45","created_at":"2026-07-06T11:48:04.388Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2ac804d85ef846c6e238f28c6bb43636","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.397Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-90e373fbefce7ff9229c3e45: start","job_status":"start","queue_duration_s":0.950857,"scheduling_latency_s":0.950933} gitlab # [ 179.649490] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:05.605Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"90e373fbefce7ff9229c3e45","created_at":"2026-07-06T11:48:04.388Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2ac804d85ef846c6e238f28c6bb43636","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.397Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-90e373fbefce7ff9229c3e45: done: 0.257332 sec","job_status":"done","queue_duration_s":0.950857,"scheduling_latency_s":0.950933,"redis_calls":12,"redis_duration_s":0.006868,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001152,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003293,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002423,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.202535,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001298769,"exclusive_lock_hold_duration_s":0.201062704,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.257332,"completed_at":"2026-07-06T11:48:05.605Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.030524,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 179.759241] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:05.714Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7f83fe1575753c4c6a599ca5","created_at":"2026-07-06T11:48:04.397Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-8719d35934f941d9","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"70cae59c3e5e6098ffe34411c6ee89b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.413Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7f83fe1575753c4c6a599ca5: start","job_status":"start","queue_duration_s":1.301467,"scheduling_latency_s":1.301515} gitlab # [ 179.811240] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:05.767Z","message":"No existing merge request to be cleaned up."} gitlab # [ 179.840209] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:05.795Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7f83fe1575753c4c6a599ca5","created_at":"2026-07-06T11:48:04.397Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-8719d35934f941d9","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"70cae59c3e5e6098ffe34411c6ee89b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.413Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7f83fe1575753c4c6a599ca5: done: 0.081009 sec","job_status":"done","queue_duration_s":1.301467,"scheduling_latency_s":1.301515,"redis_calls":13,"redis_duration_s":0.010149,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000827,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002644,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.006678,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036211,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000911289,"duration_s":0.081009,"completed_at":"2026-07-06T11:48:05.795Z","load_balancing_strategy":"primary","db_duration_s":0.009229,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 179.918227] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:05.873Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"22abb78596971e37d6a241be","created_at":"2026-07-06T11:48:04.397Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-8719d35934f941d9","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"70cae59c3e5e6098ffe34411c6ee89b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.413Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-22abb78596971e37d6a241be: start","job_status":"start","queue_duration_s":1.46005,"scheduling_latency_s":1.460219} gitlab # [ 179.945140] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:05.901Z","message":"No existing merge request to be cleaned up."} gitlab # [ 179.963747] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:05.919Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"22abb78596971e37d6a241be","created_at":"2026-07-06T11:48:04.397Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-8719d35934f941d9","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"70cae59c3e5e6098ffe34411c6ee89b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.413Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-22abb78596971e37d6a241be: done: 0.045935 sec","job_status":"done","queue_duration_s":1.46005,"scheduling_latency_s":1.460219,"redis_calls":10,"redis_duration_s":0.008455,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002037,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.006418,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.045935,"completed_at":"2026-07-06T11:48:05.919Z","load_balancing_strategy":"primary","db_duration_s":0.004788,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.026877] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:05.981Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0f2fd3cdb5c97bdf9b5366c0","created_at":"2026-07-06T11:48:04.397Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-8719d35934f941d9","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"70cae59c3e5e6098ffe34411c6ee89b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.413Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0f2fd3cdb5c97bdf9b5366c0: start","job_status":"start","queue_duration_s":1.568424,"scheduling_latency_s":1.568469} gitlab # [ 180.054863] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:06.010Z","message":"No existing merge request to be cleaned up."} gitlab # [ 180.074147] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:06.029Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0f2fd3cdb5c97bdf9b5366c0","created_at":"2026-07-06T11:48:04.397Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-8719d35934f941d9","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"70cae59c3e5e6098ffe34411c6ee89b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.413Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0f2fd3cdb5c97bdf9b5366c0: done: 0.047799 sec","job_status":"done","queue_duration_s":1.568424,"scheduling_latency_s":1.568469,"redis_calls":10,"redis_duration_s":0.008667,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002043,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.006624,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025733,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.047799,"completed_at":"2026-07-06T11:48:06.029Z","load_balancing_strategy":"primary","db_duration_s":0.004844,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.137227] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:06.092Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"60641693f8daf9538d7a682e","created_at":"2026-07-06T11:48:04.397Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-8719d35934f941d9","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"70cae59c3e5e6098ffe34411c6ee89b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.413Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-60641693f8daf9538d7a682e: start","job_status":"start","queue_duration_s":1.679451,"scheduling_latency_s":1.67952} gitlab # [ 180.164285] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:06.120Z","message":"No existing merge request to be cleaned up."} gitlab # [ 180.183194] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:06.138Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"60641693f8daf9538d7a682e","created_at":"2026-07-06T11:48:04.397Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-8719d35934f941d9","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"70cae59c3e5e6098ffe34411c6ee89b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.413Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-60641693f8daf9538d7a682e: done: 0.046021 sec","job_status":"done","queue_duration_s":1.679451,"scheduling_latency_s":1.67952,"redis_calls":10,"redis_duration_s":0.008241,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001977,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.006264,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026224,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.046021,"completed_at":"2026-07-06T11:48:06.138Z","load_balancing_strategy":"primary","db_duration_s":0.004631,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.245537] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:06.201Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"fdf53f3eeee1f7e93d8a133c","created_at":"2026-07-06T11:48:04.408Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c3f576484b54e2a7cb15d982c5304d44","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.415Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-fdf53f3eeee1f7e93d8a133c: start","job_status":"start","queue_duration_s":1.785108,"scheduling_latency_s":1.785177} gitlab # [ 180.316802] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:06.272Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"fdf53f3eeee1f7e93d8a133c","created_at":"2026-07-06T11:48:04.408Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c3f576484b54e2a7cb15d982c5304d44","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.415Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-fdf53f3eeee1f7e93d8a133c: done: 0.07126 sec","job_status":"done","queue_duration_s":1.785108,"scheduling_latency_s":1.785177,"redis_calls":12,"redis_duration_s":0.006477,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000973,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003367,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002137,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028714,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001030857,"exclusive_lock_hold_duration_s":0.009710733,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.07126,"completed_at":"2026-07-06T11:48:06.272Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00589,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.388171] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:06.342Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"8af4ad9d090943489be8e12a","created_at":"2026-07-06T11:48:04.429Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"69b9f335014c7a38c155037254f0a122","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.435Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-8af4ad9d090943489be8e12a: start","job_status":"start","queue_duration_s":1.90688,"scheduling_latency_s":1.906959} gitlab # [ 180.416422] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:06.372Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 180.431762] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:06.387Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"8af4ad9d090943489be8e12a","created_at":"2026-07-06T11:48:04.429Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"69b9f335014c7a38c155037254f0a122","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.435Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-8af4ad9d090943489be8e12a: done: 0.044876 sec","job_status":"done","queue_duration_s":1.90688,"scheduling_latency_s":1.906959,"redis_calls":10,"redis_duration_s":0.006107,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001275,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003554,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001278,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024842,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000933359,"duration_s":0.044876,"completed_at":"2026-07-06T11:48:06.387Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00126,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.501858] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:06.457Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"ddae4c9c38dfa62d9f8ce979","created_at":"2026-07-06T11:48:04.447Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"956663199d75efa279ba485312d88ad4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.455Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-ddae4c9c38dfa62d9f8ce979: start","job_status":"start","queue_duration_s":2.001356,"scheduling_latency_s":2.00142} gitlab # [ 180.594377] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:06.550Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"ddae4c9c38dfa62d9f8ce979","created_at":"2026-07-06T11:48:04.447Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"956663199d75efa279ba485312d88ad4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.455Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-ddae4c9c38dfa62d9f8ce979: done: 0.09306 sec","job_status":"done","queue_duration_s":2.001356,"scheduling_latency_s":2.00142,"redis_calls":14,"redis_duration_s":0.007765,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002259,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003012,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002494,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.02,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035815,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001074159,"exclusive_lock_hold_duration_s":0.053602623,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.09306,"completed_at":"2026-07-06T11:48:06.550Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.048728,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.666456] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:06.622Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"3a5a9dec2cb285c18155f76a","created_at":"2026-07-06T11:48:04.459Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"94906ada0bf375b6fb0f91f7c267970a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.468Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-3a5a9dec2cb285c18155f76a: start","job_status":"start","queue_duration_s":2.153808,"scheduling_latency_s":2.153885} gitlab # [ 180.724757] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:06.680Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"3a5a9dec2cb285c18155f76a","created_at":"2026-07-06T11:48:04.459Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"94906ada0bf375b6fb0f91f7c267970a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.468Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-3a5a9dec2cb285c18155f76a: done: 0.058584 sec","job_status":"done","queue_duration_s":2.153808,"scheduling_latency_s":2.153885,"redis_calls":12,"redis_duration_s":0.008656,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001234,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004655,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002767,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001428114,"exclusive_lock_hold_duration_s":0.011945093,"duration_s":0.058584,"completed_at":"2026-07-06T11:48:06.680Z","load_balancing_strategy":"primary","db_duration_s":0.006202,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.792419] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:06.747Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"eeb766bb4ca26457964cfcc5","created_at":"2026-07-06T11:48:04.473Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.478Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-eeb766bb4ca26457964cfcc5: start","job_status":"start","queue_duration_s":2.269372,"scheduling_latency_s":2.269446} gitlab # [ 180.938458] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:06.894Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"eeb766bb4ca26457964cfcc5","created_at":"2026-07-06T11:48:04.473Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.478Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-eeb766bb4ca26457964cfcc5: done: 0.146394 sec","job_status":"done","queue_duration_s":2.269372,"scheduling_latency_s":2.269446,"redis_calls":71,"redis_duration_s":0.036285,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.005777,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001356,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.027445,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001707,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.101758,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000541969,"duration_s":0.146394,"completed_at":"2026-07-06T11:48:06.894Z","load_balancing_strategy":"primary","db_duration_s":0.004592,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 181.029151] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:06.984Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"7823eb298f97005188e30c26","created_at":"2026-07-06T11:48:04.488Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d289e3b3cf6fd59ad0bdd7ce514f7da","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.494Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-7823eb298f97005188e30c26: start","job_status":"start","queue_duration_s":2.490777,"scheduling_latency_s":2.490825} gitlab # [ 181.097110] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.052Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"7823eb298f97005188e30c26","created_at":"2026-07-06T11:48:04.488Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d289e3b3cf6fd59ad0bdd7ce514f7da","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.494Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-7823eb298f97005188e30c26: done: 0.067305 sec","job_status":"done","queue_duration_s":2.490777,"scheduling_latency_s":2.490825,"redis_calls":10,"redis_duration_s":0.0041470000000000005,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000778,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002533,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000836,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028218,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000753448,"duration_s":0.067305,"completed_at":"2026-07-06T11:48:07.052Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00508,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 181.167565] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.123Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"2dac15b17c4373f4c42e036c","created_at":"2026-07-06T11:48:04.496Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b15ef0b3085b5e0742b26904ac9a1035","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-06T11:48:04.502Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-2dac15b17c4373f4c42e036c: start","job_status":"start","queue_duration_s":2.621064,"scheduling_latency_s":2.621123} gitlab # [ 181.249625] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.205Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"2dac15b17c4373f4c42e036c","created_at":"2026-07-06T11:48:04.496Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b15ef0b3085b5e0742b26904ac9a1035","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-06T11:48:04.502Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-2dac15b17c4373f4c42e036c: done: 0.081849 sec","job_status":"done","queue_duration_s":2.621064,"scheduling_latency_s":2.621123,"redis_calls":10,"redis_duration_s":0.004087,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00071,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002512,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000865,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037192,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000882794,"duration_s":0.081849,"completed_at":"2026-07-06T11:48:07.205Z","load_balancing_strategy":"primary","db_duration_s":0.006786,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 181.326981] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.282Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"6f4107e6fc05625baa37ae50","created_at":"2026-07-06T11:48:04.516Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3204973c66957a6a214b708fe9c5f6aa","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.521Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-6f4107e6fc05625baa37ae50: start","job_status":"start","queue_duration_s":2.761266,"scheduling_latency_s":2.761445} gitlab # [ 181.392665] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.347Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"6f4107e6fc05625baa37ae50","created_at":"2026-07-06T11:48:04.516Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3204973c66957a6a214b708fe9c5f6aa","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.521Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-6f4107e6fc05625baa37ae50: done: 0.064715 sec","job_status":"done","queue_duration_s":2.761266,"scheduling_latency_s":2.761445,"redis_calls":10,"redis_duration_s":0.006562999999999999,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001354,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003913,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001296,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029497,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001015213,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.064715,"completed_at":"2026-07-06T11:48:07.346Z","load_balancing_strategy":"primary","db_duration_s":0.016798,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 181.464672] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.420Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"82e538d6323e9c14e89bdf9b","created_at":"2026-07-06T11:48:04.531Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e1488eb3e21b34af3779bdf09702421b","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.561Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-82e538d6323e9c14e89bdf9b: start","job_status":"start","queue_duration_s":2.858324,"scheduling_latency_s":2.858401} gitlab # [ 181.511257] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.466Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"82e538d6323e9c14e89bdf9b","created_at":"2026-07-06T11:48:04.531Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e1488eb3e21b34af3779bdf09702421b","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.561Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-82e538d6323e9c14e89bdf9b: done: 0.047039 sec","job_status":"done","queue_duration_s":2.858324,"scheduling_latency_s":2.858401,"redis_calls":12,"redis_duration_s":0.007859,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.001526,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001275,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003892,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001166,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026082,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000961575,"duration_s":0.047039,"completed_at":"2026-07-06T11:48:07.466Z","load_balancing_strategy":"primary","db_duration_s":0.001357,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 181.580839] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.536Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"854437fb5ee74374a0cb140f","created_at":"2026-07-06T11:48:04.598Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2cd814781ebd2562d41961c6b05eb8c9","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.607Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-854437fb5ee74374a0cb140f: start","job_status":"start","queue_duration_s":2.929154,"scheduling_latency_s":2.929228} gitlab # [ 181.635674] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.591Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"854437fb5ee74374a0cb140f","created_at":"2026-07-06T11:48:04.598Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2cd814781ebd2562d41961c6b05eb8c9","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.607Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-854437fb5ee74374a0cb140f: done: 0.055342 sec","job_status":"done","queue_duration_s":2.929154,"scheduling_latency_s":2.929228,"redis_calls":13,"redis_duration_s":0.007122,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001338,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003302,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002482,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032326,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.013797563,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.055342,"completed_at":"2026-07-06T11:48:07.591Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005994,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 181.672635] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.594Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"9c75ee9f331ff756240bc617","created_at":"2026-07-06T11:48:04.614Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bcfd83af0e699a0c170ebc5433d6d71c","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.620Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-9c75ee9f331ff756240bc617: start","job_status":"start","queue_duration_s":2.973894,"scheduling_latency_s":2.97392} gitlab # [ 181.698884] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.654Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"9c75ee9f331ff756240bc617","created_at":"2026-07-06T11:48:04.614Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bcfd83af0e699a0c170ebc5433d6d71c","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.620Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-9c75ee9f331ff756240bc617: done: 0.060492 sec","job_status":"done","queue_duration_s":2.973894,"scheduling_latency_s":2.97392,"redis_calls":12,"redis_duration_s":0.005091,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001002,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002445,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001644,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04303,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000977219,"exclusive_lock_hold_duration_s":0.007186668,"duration_s":0.060492,"completed_at":"2026-07-06T11:48:07.654Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003429,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 181.723057] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.658Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"a306aa2de4f0d8ec86dbfffd","created_at":"2026-07-06T11:48:04.625Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"54ac0ce08388d84f452c96e2de80602c","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.632Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-a306aa2de4f0d8ec86dbfffd: start","job_status":"start","queue_duration_s":3.025207,"scheduling_latency_s":3.025248} gitlab # [ 181.745068] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.680Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"a306aa2de4f0d8ec86dbfffd","created_at":"2026-07-06T11:48:04.625Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"54ac0ce08388d84f452c96e2de80602c","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.632Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-a306aa2de4f0d8ec86dbfffd: done: 0.022915 sec","job_status":"done","queue_duration_s":3.025207,"scheduling_latency_s":3.025248,"redis_calls":10,"redis_duration_s":0.0027990000000000003,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00044,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001623,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000736,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011699,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000410108,"duration_s":0.022915,"completed_at":"2026-07-06T11:48:07.680Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001296,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 181.810415] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.682Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"343a948e39aac5ab87c37386","created_at":"2026-07-06T11:48:04.675Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2a800e1aaa8003cda7f1b30b9ee4e417","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.685Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-343a948e39aac5ab87c37386: start","job_status":"start","queue_duration_s":2.996912,"scheduling_latency_s":2.996932} gitlab # [ 181.878631] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.692Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 181.880508] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.698Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"343a948e39aac5ab87c37386","created_at":"2026-07-06T11:48:04.675Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2a800e1aaa8003cda7f1b30b9ee4e417","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.685Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-343a948e39aac5ab87c37386: done: 0.0165 sec","job_status":"done","queue_duration_s":2.996912,"scheduling_latency_s":2.996932,"redis_calls":10,"redis_duration_s":0.002343,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00044,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001376,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000527,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009351,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000355352,"duration_s":0.0165,"completed_at":"2026-07-06T11:48:07.698Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0006,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 181.902521] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.700Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"ed68f6a0d457200be2f73b92","created_at":"2026-07-06T11:48:04.696Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f3fe3b10bb3da789c325da5cf9a5f527","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-06T11:48:04.704Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-ed68f6a0d457200be2f73b92: start","job_status":"start","queue_duration_s":2.99552,"scheduling_latency_s":2.99554} gitlab # [ 181.923524] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.710Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 181.928535] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.716Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"ed68f6a0d457200be2f73b92","created_at":"2026-07-06T11:48:04.696Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f3fe3b10bb3da789c325da5cf9a5f527","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-06T11:48:04.704Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-ed68f6a0d457200be2f73b92: done: 0.016733 sec","job_status":"done","queue_duration_s":2.99552,"scheduling_latency_s":2.99554,"redis_calls":10,"redis_duration_s":0.002459,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000467,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001519,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000473,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009397,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000398654,"duration_s":0.016733,"completed_at":"2026-07-06T11:48:07.716Z","load_balancing_strategy":"primary","db_duration_s":0.000871,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.019180] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.718Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"2e9aee13927859ad2b554658","created_at":"2026-07-06T11:48:04.714Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"315b337094518ff7ad9dbdd47d1d26b9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-06T11:48:04.724Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-2e9aee13927859ad2b554658: start","job_status":"start","queue_duration_s":2.993701,"scheduling_latency_s":2.993722} gitlab # [ 182.020283] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.729Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 182.021470] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.735Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"2e9aee13927859ad2b554658","created_at":"2026-07-06T11:48:04.714Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"315b337094518ff7ad9dbdd47d1d26b9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-06T11:48:04.724Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-2e9aee13927859ad2b554658: done: 0.01682 sec","job_status":"done","queue_duration_s":2.993701,"scheduling_latency_s":2.993722,"redis_calls":10,"redis_duration_s":0.002271,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000436,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001392,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000443,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00924,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000352838,"duration_s":0.01682,"completed_at":"2026-07-06T11:48:07.735Z","load_balancing_strategy":"primary","db_duration_s":0.00058,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.104357] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.736Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"cf16e37a909eb03cdb32c06d","created_at":"2026-07-06T11:48:04.734Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8852b1271ada3787c7fb2cf6ec166cc1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.741Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-cf16e37a909eb03cdb32c06d: start","job_status":"start","queue_duration_s":2.994927,"scheduling_latency_s":2.994948} gitlab # [ 182.105824] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.754Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"cf16e37a909eb03cdb32c06d","created_at":"2026-07-06T11:48:04.734Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8852b1271ada3787c7fb2cf6ec166cc1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.741Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-cf16e37a909eb03cdb32c06d: done: 0.018265 sec","job_status":"done","queue_duration_s":2.994927,"scheduling_latency_s":2.994948,"redis_calls":10,"redis_duration_s":0.0023020000000000002,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00044,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001374,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000488,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000345575,"duration_s":0.018265,"completed_at":"2026-07-06T11:48:07.754Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001171,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.169552] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.756Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"b6e5a8228258eaccd12ac1c3","created_at":"2026-07-06T11:48:04.763Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"15639d73c889a2956b34b7a8a2389ab8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.771Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-b6e5a8228258eaccd12ac1c3: start","job_status":"start","queue_duration_s":2.985169,"scheduling_latency_s":2.985189} gitlab # [ 182.192350] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.766Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 182.198513] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.773Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"b6e5a8228258eaccd12ac1c3","created_at":"2026-07-06T11:48:04.763Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"15639d73c889a2956b34b7a8a2389ab8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.771Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-b6e5a8228258eaccd12ac1c3: done: 0.017345 sec","job_status":"done","queue_duration_s":2.985169,"scheduling_latency_s":2.985189,"redis_calls":10,"redis_duration_s":0.002545,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000435,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001672,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000438,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00953,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000659581,"duration_s":0.017345,"completed_at":"2026-07-06T11:48:07.773Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000435,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.300598] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.775Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"add0a05a556d70e575dff004","created_at":"2026-07-06T11:48:04.787Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c7b216fefe74025df153bb758c560359","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.794Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-add0a05a556d70e575dff004: start","job_status":"start","queue_duration_s":2.980274,"scheduling_latency_s":2.980295} gitlab # [ 182.302174] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.793Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"add0a05a556d70e575dff004","created_at":"2026-07-06T11:48:04.787Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c7b216fefe74025df153bb758c560359","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.794Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-add0a05a556d70e575dff004: done: 0.018796 sec","job_status":"done","queue_duration_s":2.980274,"scheduling_latency_s":2.980295,"redis_calls":12,"redis_duration_s":0.0029760000000000003,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000423,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001492,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001061,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000399212,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.018796,"completed_at":"2026-07-06T11:48:07.793Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000468,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.358354] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:07.795Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"c8d5c97053fcf915f97fd9c8","created_at":"2026-07-06T11:48:04.816Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"385427ae563fa86cdc96ebf9abbe3d25","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.823Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-c8d5c97053fcf915f97fd9c8: start","job_status":"start","queue_duration_s":2.971341,"scheduling_latency_s":2.971362} gitlab # [ 182.442956] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.002Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"c8d5c97053fcf915f97fd9c8","created_at":"2026-07-06T11:48:04.816Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"385427ae563fa86cdc96ebf9abbe3d25","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.823Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-c8d5c97053fcf915f97fd9c8: done: 0.207578 sec","job_status":"done","queue_duration_s":2.971341,"scheduling_latency_s":2.971362,"redis_calls":14,"redis_duration_s":0.0037819999999999998,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000991,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001546,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001245,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":125,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":125,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.036,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":125,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":80,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.036,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.134948,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000454248,"exclusive_lock_hold_duration_s":0.184683249,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.207578,"completed_at":"2026-07-06T11:48:08.002Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.075609,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.448178] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.004Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"c3670f7abdede630e313f030","created_at":"2026-07-06T11:48:04.834Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"430c3c5bed8f9c74e06b03430a5f7cd1","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.843Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-c3670f7abdede630e313f030: start","job_status":"start","queue_duration_s":3.161173,"scheduling_latency_s":3.161197} gitlab # [ 182.471421] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.031Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"c3670f7abdede630e313f030","created_at":"2026-07-06T11:48:04.834Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"430c3c5bed8f9c74e06b03430a5f7cd1","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.843Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-c3670f7abdede630e313f030: done: 0.026303 sec","job_status":"done","queue_duration_s":3.161173,"scheduling_latency_s":3.161197,"redis_calls":10,"redis_duration_s":0.002624,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000494,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001694,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000436,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013762,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000415696,"duration_s":0.026303,"completed_at":"2026-07-06T11:48:08.030Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004508,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.584497] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.032Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"bd7534dfec3451575558f30f","created_at":"2026-07-06T11:48:04.854Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.862Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-bd7534dfec3451575558f30f: start","job_status":"start","queue_duration_s":3.169484,"scheduling_latency_s":3.169506} gitlab # [ 182.585969] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.117Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"bd7534dfec3451575558f30f","created_at":"2026-07-06T11:48:04.854Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-c82878938a864b4c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:04.862Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-bd7534dfec3451575558f30f: done: 0.085311 sec","job_status":"done","queue_duration_s":3.169484,"scheduling_latency_s":3.169506,"redis_calls":65,"redis_duration_s":0.016055,"redis_read_bytes":115,"redis_write_bytes":41282,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001002,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":656,"redis_queues_calls":2,"redis_queues_duration_s":0.000663,"redis_queues_read_bytes":4,"redis_queues_write_bytes":36985,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.013365,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3374,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001025,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":267,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.065946,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000438882,"duration_s":0.085311,"completed_at":"2026-07-06T11:48:08.117Z","load_balancing_strategy":"primary","db_duration_s":0.000794,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.686238] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.119Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0447fe34f1f0f39ef80b5439","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0447fe34f1f0f39ef80b5439: start","job_status":"start","queue_duration_s":1.234821,"scheduling_latency_s":1.234844} gitlab # [ 182.687474] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.141Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0447fe34f1f0f39ef80b5439","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0447fe34f1f0f39ef80b5439: done: 0.022432 sec","job_status":"done","queue_duration_s":1.234821,"scheduling_latency_s":1.234844,"redis_calls":16,"redis_duration_s":0.004350000000000001,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000457,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00115,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002743,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012745,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000404521,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022432,"completed_at":"2026-07-06T11:48:08.141Z","load_balancing_strategy":"primary","db_duration_s":0.001267,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.706088] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.143Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd6f22cdcc2350082f4ed293","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd6f22cdcc2350082f4ed293: start","job_status":"start","queue_duration_s":1.258678,"scheduling_latency_s":1.258699} gitlab # [ 182.726082] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.158Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd6f22cdcc2350082f4ed293","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd6f22cdcc2350082f4ed293: done: 0.015487 sec","job_status":"done","queue_duration_s":1.258678,"scheduling_latency_s":1.258699,"redis_calls":13,"redis_duration_s":0.0032040000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001007,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002197,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009645,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015487,"completed_at":"2026-07-06T11:48:08.158Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.787282] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.159Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec75ae4a8f76fbb948e409d9","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec75ae4a8f76fbb948e409d9: start","job_status":"start","queue_duration_s":1.275464,"scheduling_latency_s":1.275487} gitlab # [ 182.865380] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.175Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec75ae4a8f76fbb948e409d9","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec75ae4a8f76fbb948e409d9: done: 0.015176 sec","job_status":"done","queue_duration_s":1.275464,"scheduling_latency_s":1.275487,"redis_calls":13,"redis_duration_s":0.00311,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000983,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002127,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009605,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015176,"completed_at":"2026-07-06T11:48:08.175Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.867068] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.176Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f7826dc306566c5a38c0c1b","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f7826dc306566c5a38c0c1b: start","job_status":"start","queue_duration_s":1.291938,"scheduling_latency_s":1.291959} gitlab # [ 182.884838] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.191Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f7826dc306566c5a38c0c1b","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f7826dc306566c5a38c0c1b: done: 0.014937 sec","job_status":"done","queue_duration_s":1.291938,"scheduling_latency_s":1.291959,"redis_calls":13,"redis_duration_s":0.003172,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000886,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002286,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009311,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014937,"completed_at":"2026-07-06T11:48:08.191Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.005631] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.192Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9fb7fffadff6f47e1995d049","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9fb7fffadff6f47e1995d049: start","job_status":"start","queue_duration_s":1.308315,"scheduling_latency_s":1.308338} gitlab # [ 183.007205] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.208Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9fb7fffadff6f47e1995d049","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9fb7fffadff6f47e1995d049: done: 0.015283 sec","job_status":"done","queue_duration_s":1.308315,"scheduling_latency_s":1.308338,"redis_calls":13,"redis_duration_s":0.003026,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000967,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002059,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009569,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015283,"completed_at":"2026-07-06T11:48:08.208Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.024928] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.209Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8058b84d6a4809f7c786bde4","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8058b84d6a4809f7c786bde4: start","job_status":"start","queue_duration_s":1.324884,"scheduling_latency_s":1.324914} gitlab # [ 183.100188] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.224Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8058b84d6a4809f7c786bde4","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8058b84d6a4809f7c786bde4: done: 0.015147 sec","job_status":"done","queue_duration_s":1.324884,"scheduling_latency_s":1.324914,"redis_calls":13,"redis_duration_s":0.0030930000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00093,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002163,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009554,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015147,"completed_at":"2026-07-06T11:48:08.224Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.104052] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.225Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d49fe4a7acaa9673b6bfff5","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d49fe4a7acaa9673b6bfff5: start","job_status":"start","queue_duration_s":1.341394,"scheduling_latency_s":1.341416} gitlab # [ 183.174046] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.240Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d49fe4a7acaa9673b6bfff5","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d49fe4a7acaa9673b6bfff5: done: 0.015061 sec","job_status":"done","queue_duration_s":1.341394,"scheduling_latency_s":1.341416,"redis_calls":13,"redis_duration_s":0.003251,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000941,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00231,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009285,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015061,"completed_at":"2026-07-06T11:48:08.240Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.182086] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.242Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78ba290faedc8c79f997901a","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78ba290faedc8c79f997901a: start","job_status":"start","queue_duration_s":1.357743,"scheduling_latency_s":1.357765} gitlab # [ 183.247247] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.257Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78ba290faedc8c79f997901a","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78ba290faedc8c79f997901a: done: 0.015124 sec","job_status":"done","queue_duration_s":1.357743,"scheduling_latency_s":1.357765,"redis_calls":13,"redis_duration_s":0.002994,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002068,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009571,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015124,"completed_at":"2026-07-06T11:48:08.257Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.259776] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.258Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a17d5b90ac22744d32a7ff56","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a17d5b90ac22744d32a7ff56: start","job_status":"start","queue_duration_s":1.374325,"scheduling_latency_s":1.374346} gitlab # [ 183.279234] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.273Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a17d5b90ac22744d32a7ff56","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a17d5b90ac22744d32a7ff56: done: 0.014855 sec","job_status":"done","queue_duration_s":1.374325,"scheduling_latency_s":1.374346,"redis_calls":13,"redis_duration_s":0.003066,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000923,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002143,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009421,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014855,"completed_at":"2026-07-06T11:48:08.273Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.340216] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.274Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52811e18ec2eeecd69b0a501","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52811e18ec2eeecd69b0a501: start","job_status":"start","queue_duration_s":1.390474,"scheduling_latency_s":1.390496} gitlab # [ 183.422436] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.290Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52811e18ec2eeecd69b0a501","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52811e18ec2eeecd69b0a501: done: 0.015146 sec","job_status":"done","queue_duration_s":1.390474,"scheduling_latency_s":1.390496,"redis_calls":13,"redis_duration_s":0.0030110000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000929,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002082,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0095,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015146,"completed_at":"2026-07-06T11:48:08.290Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.424189] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.291Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d2863ecc92e05d20bec06fa","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d2863ecc92e05d20bec06fa: start","job_status":"start","queue_duration_s":1.407059,"scheduling_latency_s":1.40708} gitlab # [ 183.437516] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.306Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d2863ecc92e05d20bec06fa","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d2863ecc92e05d20bec06fa: done: 0.015245 sec","job_status":"done","queue_duration_s":1.407059,"scheduling_latency_s":1.40708,"redis_calls":13,"redis_duration_s":0.003079,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000954,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002125,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009593,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015245,"completed_at":"2026-07-06T11:48:08.306Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.562380] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.308Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed7d2017ec8b5cf95582a2af","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed7d2017ec8b5cf95582a2af: start","job_status":"start","queue_duration_s":1.42359,"scheduling_latency_s":1.423613} gitlab # [ 183.563578] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.323Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed7d2017ec8b5cf95582a2af","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed7d2017ec8b5cf95582a2af: done: 0.015039 sec","job_status":"done","queue_duration_s":1.42359,"scheduling_latency_s":1.423613,"redis_calls":13,"redis_duration_s":0.002981,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000933,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002048,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009428,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015039,"completed_at":"2026-07-06T11:48:08.323Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.576129] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.324Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44ff97831f8178146be2ac3e","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44ff97831f8178146be2ac3e: start","job_status":"start","queue_duration_s":1.43999,"scheduling_latency_s":1.440054} gitlab # [ 183.595542] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.339Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44ff97831f8178146be2ac3e","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44ff97831f8178146be2ac3e: done: 0.015425 sec","job_status":"done","queue_duration_s":1.43999,"scheduling_latency_s":1.440054,"redis_calls":13,"redis_duration_s":0.003123,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000939,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002184,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009369,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015425,"completed_at":"2026-07-06T11:48:08.339Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.701910] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.341Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a4fbdc0d2b8d94f7da75273","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a4fbdc0d2b8d94f7da75273: start","job_status":"start","queue_duration_s":1.45672,"scheduling_latency_s":1.456741} gitlab # [ 183.703205] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.356Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a4fbdc0d2b8d94f7da75273","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a4fbdc0d2b8d94f7da75273: done: 0.015082 sec","job_status":"done","queue_duration_s":1.45672,"scheduling_latency_s":1.456741,"redis_calls":13,"redis_duration_s":0.002934,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000983,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001951,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009471,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015082,"completed_at":"2026-07-06T11:48:08.356Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.735157] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.357Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"51b0c718c938d4486e9bc1d0","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-51b0c718c938d4486e9bc1d0: start","job_status":"start","queue_duration_s":1.473152,"scheduling_latency_s":1.473174} gitlab # [ 183.754536] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.372Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"51b0c718c938d4486e9bc1d0","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-51b0c718c938d4486e9bc1d0: done: 0.014971 sec","job_status":"done","queue_duration_s":1.473152,"scheduling_latency_s":1.473174,"redis_calls":13,"redis_duration_s":0.002987,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000944,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002043,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009414,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014971,"completed_at":"2026-07-06T11:48:08.372Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.842094] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.373Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b10201518557973c5d627c2","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b10201518557973c5d627c2: start","job_status":"start","queue_duration_s":1.489414,"scheduling_latency_s":1.489436} gitlab # [ 183.843523] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.389Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b10201518557973c5d627c2","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b10201518557973c5d627c2: done: 0.015358 sec","job_status":"done","queue_duration_s":1.489414,"scheduling_latency_s":1.489436,"redis_calls":13,"redis_duration_s":0.003097,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000951,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002146,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009544,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015358,"completed_at":"2026-07-06T11:48:08.389Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.894157] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.390Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2fcacad900ad3e405ba03f3","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2fcacad900ad3e405ba03f3: start","job_status":"start","queue_duration_s":1.506309,"scheduling_latency_s":1.506331} gitlab # [ 183.913460] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.406Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2fcacad900ad3e405ba03f3","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2fcacad900ad3e405ba03f3: done: 0.015345 sec","job_status":"done","queue_duration_s":1.506309,"scheduling_latency_s":1.506331,"redis_calls":13,"redis_duration_s":0.003045,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000961,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002084,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009561,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015345,"completed_at":"2026-07-06T11:48:08.406Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.982213] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.407Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e57a5e09fb79a8a6cc6c44a","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e57a5e09fb79a8a6cc6c44a: start","job_status":"start","queue_duration_s":1.522946,"scheduling_latency_s":1.522968} gitlab # [ 183.992053] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.422Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e57a5e09fb79a8a6cc6c44a","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e57a5e09fb79a8a6cc6c44a: done: 0.014968 sec","job_status":"done","queue_duration_s":1.522946,"scheduling_latency_s":1.522968,"redis_calls":13,"redis_duration_s":0.0030310000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000935,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009397,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014968,"completed_at":"2026-07-06T11:48:08.422Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 184.072123] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.423Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee363839dacbeaae89dd1f4e","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee363839dacbeaae89dd1f4e: start","job_status":"start","queue_duration_s":1.539363,"scheduling_latency_s":1.539387} gitlab # [ 184.073465] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.438Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee363839dacbeaae89dd1f4e","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee363839dacbeaae89dd1f4e: done: 0.014762 sec","job_status":"done","queue_duration_s":1.539363,"scheduling_latency_s":1.539387,"redis_calls":13,"redis_duration_s":0.002949,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00096,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001989,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009269,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014762,"completed_at":"2026-07-06T11:48:08.438Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 184.132950] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.440Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00955b195e85f923f1031a40","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00955b195e85f923f1031a40: start","job_status":"start","queue_duration_s":1.555606,"scheduling_latency_s":1.555627} gitlab # [ 184.217197] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.455Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00955b195e85f923f1031a40","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00955b195e85f923f1031a40: done: 0.015428 sec","job_status":"done","queue_duration_s":1.555606,"scheduling_latency_s":1.555627,"redis_calls":13,"redis_duration_s":0.003019,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000967,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002052,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009598,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015428,"completed_at":"2026-07-06T11:48:08.455Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 184.218986] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.456Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c092962ba0f955a06d2ca157","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c092962ba0f955a06d2ca157: start","job_status":"start","queue_duration_s":1.572378,"scheduling_latency_s":1.572399} gitlab # [ 184.232173] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.472Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c092962ba0f955a06d2ca157","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c092962ba0f955a06d2ca157: done: 0.015152 sec","job_status":"done","queue_duration_s":1.572378,"scheduling_latency_s":1.572399,"redis_calls":13,"redis_duration_s":0.0031260000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000955,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002171,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009548,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015152,"completed_at":"2026-07-06T11:48:08.471Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 184.294293] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.473Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12def61109152475d28a56d6","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12def61109152475d28a56d6: start","job_status":"start","queue_duration_s":1.588872,"scheduling_latency_s":1.588895} gitlab # [ 184.313359] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.488Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12def61109152475d28a56d6","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12def61109152475d28a56d6: done: 0.015347 sec","job_status":"done","queue_duration_s":1.588872,"scheduling_latency_s":1.588895,"redis_calls":13,"redis_duration_s":0.0031739999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000927,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002247,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009577,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015347,"completed_at":"2026-07-06T11:48:08.488Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 184.405280] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.490Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"256f830b435377af990bd05b","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-256f830b435377af990bd05b: start","job_status":"start","queue_duration_s":1.605538,"scheduling_latency_s":1.605562} gitlab # [ 184.406856] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.505Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"256f830b435377af990bd05b","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-256f830b435377af990bd05b: done: 0.015219 sec","job_status":"done","queue_duration_s":1.605538,"scheduling_latency_s":1.605562,"redis_calls":13,"redis_duration_s":0.003092,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000974,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002118,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009507,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015219,"completed_at":"2026-07-06T11:48:08.505Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 184.453212] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.506Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f29b2b8357bee8b32eddcff","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f29b2b8357bee8b32eddcff: start","job_status":"start","queue_duration_s":1.622173,"scheduling_latency_s":1.622196} gitlab # [ 184.472532] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.521Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f29b2b8357bee8b32eddcff","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f29b2b8357bee8b32eddcff: done: 0.014773 sec","job_status":"done","queue_duration_s":1.622173,"scheduling_latency_s":1.622196,"redis_calls":13,"redis_duration_s":0.003018,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000928,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00209,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009331,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014773,"completed_at":"2026-07-06T11:48:08.521Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 184.545334] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.522Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1a6472afc8c227dec460ba1","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1a6472afc8c227dec460ba1: start","job_status":"start","queue_duration_s":1.638382,"scheduling_latency_s":1.638402} gitlab # [ 184.550548] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.538Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1a6472afc8c227dec460ba1","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1a6472afc8c227dec460ba1: done: 0.015851 sec","job_status":"done","queue_duration_s":1.638382,"scheduling_latency_s":1.638402,"redis_calls":13,"redis_duration_s":0.0037589999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001669,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00209,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00955,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015851,"completed_at":"2026-07-06T11:48:08.538Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 184.612804] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.540Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a20b631763f563a3cb83998","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a20b631763f563a3cb83998: start","job_status":"start","queue_duration_s":1.655816,"scheduling_latency_s":1.655838} gitlab # [ 184.685320] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.555Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a20b631763f563a3cb83998","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a20b631763f563a3cb83998: done: 0.015348 sec","job_status":"done","queue_duration_s":1.655816,"scheduling_latency_s":1.655838,"redis_calls":13,"redis_duration_s":0.003045,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000937,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002108,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00953,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015348,"completed_at":"2026-07-06T11:48:08.555Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 184.690448] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:08.557Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"628974a4773613ac671f4763","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-628974a4773613ac671f4763: start","job_status":"start","queue_duration_s":1.672606,"scheduling_latency_s":1.67263} gitlab # [ 185.031357] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 2515675093 wd_nsec: 2515674718 gitlab # [ 184.797308] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:10.752Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"628974a4773613ac671f4763","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-628974a4773613ac671f4763: done: 2.195837 sec","job_status":"done","queue_duration_s":1.672606,"scheduling_latency_s":1.67263,"redis_calls":13,"redis_duration_s":0.003515,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002006,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012586,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":2.195837,"completed_at":"2026-07-06T11:48:10.752Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 184.832978] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:10.756Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f719582e57e837deeb40bd8","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f719582e57e837deeb40bd8: start","job_status":"start","queue_duration_s":3.871688,"scheduling_latency_s":3.871742} gitlab # [ 184.834203] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:10.786Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f719582e57e837deeb40bd8","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f719582e57e837deeb40bd8: done: 0.030649 sec","job_status":"done","queue_duration_s":3.871688,"scheduling_latency_s":3.871742,"redis_calls":13,"redis_duration_s":0.004899,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001832,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003067,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019842,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.030649,"completed_at":"2026-07-06T11:48:10.786Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 184.868494] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:10.788Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"082667d18a85c5bfb7d29a3a","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-082667d18a85c5bfb7d29a3a: start","job_status":"start","queue_duration_s":3.9036,"scheduling_latency_s":3.903623} gitlab # [ 184.984123] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:10.939Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"082667d18a85c5bfb7d29a3a","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-082667d18a85c5bfb7d29a3a: done: 0.151418 sec","job_status":"done","queue_duration_s":3.9036,"scheduling_latency_s":3.903623,"redis_calls":13,"redis_duration_s":0.00546,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00188,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00358,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018933,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.151418,"completed_at":"2026-07-06T11:48:10.939Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 185.020916] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:10.943Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ebf2e8e259a86600a431500","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ebf2e8e259a86600a431500: start","job_status":"start","queue_duration_s":4.058739,"scheduling_latency_s":4.058791} gitlab # [ 185.022517] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:10.973Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ebf2e8e259a86600a431500","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ebf2e8e259a86600a431500: done: 0.02988 sec","job_status":"done","queue_duration_s":4.058739,"scheduling_latency_s":4.058791,"redis_calls":13,"redis_duration_s":0.006947999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002128,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00482,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018235,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02988,"completed_at":"2026-07-06T11:48:10.973Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 185.057850] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:10.975Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b2ef9a28d971483933083f1","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b2ef9a28d971483933083f1: start","job_status":"start","queue_duration_s":4.090512,"scheduling_latency_s":4.090535} gitlab # [ 185.180948] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.136Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b2ef9a28d971483933083f1","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b2ef9a28d971483933083f1: done: 0.161099 sec","job_status":"done","queue_duration_s":4.090512,"scheduling_latency_s":4.090535,"redis_calls":13,"redis_duration_s":0.007939,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005377,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02173,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.161099,"completed_at":"2026-07-06T11:48:11.136Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 185.243617] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.199Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed3d38d5261adf22a2dcc5a2","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed3d38d5261adf22a2dcc5a2: start","job_status":"start","queue_duration_s":4.314576,"scheduling_latency_s":4.314646} gitlab # [ 185.300582] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.256Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed3d38d5261adf22a2dcc5a2","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed3d38d5261adf22a2dcc5a2: done: 0.057059 sec","job_status":"done","queue_duration_s":4.314576,"scheduling_latency_s":4.314646,"redis_calls":13,"redis_duration_s":0.008154,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00252,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005634,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02435,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.057059,"completed_at":"2026-07-06T11:48:11.255Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 185.365396] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.321Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"250a2151ab6ce1e1bcf5975a","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-250a2151ab6ce1e1bcf5975a: start","job_status":"start","queue_duration_s":4.436721,"scheduling_latency_s":4.436766} gitlab # [ 185.432625] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.388Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"250a2151ab6ce1e1bcf5975a","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-250a2151ab6ce1e1bcf5975a: done: 0.06698 sec","job_status":"done","queue_duration_s":4.436721,"scheduling_latency_s":4.436766,"redis_calls":13,"redis_duration_s":0.017319,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002887,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.014432,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017147,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.06698,"completed_at":"2026-07-06T11:48:11.388Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 185.493381] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.392Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"69c996fdb94d70d73b57c38c","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-69c996fdb94d70d73b57c38c: start","job_status":"start","queue_duration_s":4.507445,"scheduling_latency_s":4.507505} gitlab # [ 185.495223] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.427Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"69c996fdb94d70d73b57c38c","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-69c996fdb94d70d73b57c38c: done: 0.035934 sec","job_status":"done","queue_duration_s":4.507445,"scheduling_latency_s":4.507505,"redis_calls":13,"redis_duration_s":0.0076029999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002646,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004957,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021755,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035934,"completed_at":"2026-07-06T11:48:11.427Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 185.505970] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.430Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d73d0e0f06020ca42cf333c","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d73d0e0f06020ca42cf333c: start","job_status":"start","queue_duration_s":4.546285,"scheduling_latency_s":4.54633} gitlab # [ 185.526329] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.453Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d73d0e0f06020ca42cf333c","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d73d0e0f06020ca42cf333c: done: 0.022279 sec","job_status":"done","queue_duration_s":4.546285,"scheduling_latency_s":4.54633,"redis_calls":13,"redis_duration_s":0.0053609999999999994,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001351,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00401,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022279,"completed_at":"2026-07-06T11:48:11.452Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 185.588285] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.454Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6311598ef7f5d2c7fab10812","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6311598ef7f5d2c7fab10812: start","job_status":"start","queue_duration_s":4.569891,"scheduling_latency_s":4.569923} gitlab # [ 185.608306] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.470Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6311598ef7f5d2c7fab10812","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6311598ef7f5d2c7fab10812: done: 0.015858 sec","job_status":"done","queue_duration_s":4.569891,"scheduling_latency_s":4.569923,"redis_calls":13,"redis_duration_s":0.003264,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000932,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002332,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009892,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015858,"completed_at":"2026-07-06T11:48:11.470Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 185.707487] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.471Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f3d0f11d6a2dbccf525bbf0","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f3d0f11d6a2dbccf525bbf0: start","job_status":"start","queue_duration_s":4.587117,"scheduling_latency_s":4.58714} gitlab # [ 185.709052] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.486Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f3d0f11d6a2dbccf525bbf0","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f3d0f11d6a2dbccf525bbf0: done: 0.015248 sec","job_status":"done","queue_duration_s":4.587117,"scheduling_latency_s":4.58714,"redis_calls":13,"redis_duration_s":0.0031539999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001012,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002142,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009546,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015248,"completed_at":"2026-07-06T11:48:11.486Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 185.749151] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.488Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff13aba753b15268dc5664d0","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff13aba753b15268dc5664d0: start","job_status":"start","queue_duration_s":4.603846,"scheduling_latency_s":4.603868} gitlab # [ 185.768117] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.503Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff13aba753b15268dc5664d0","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff13aba753b15268dc5664d0: done: 0.01557 sec","job_status":"done","queue_duration_s":4.603846,"scheduling_latency_s":4.603868,"redis_calls":13,"redis_duration_s":0.003097,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000884,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002213,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00976,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01557,"completed_at":"2026-07-06T11:48:11.503Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 185.828528] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.505Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"da30af73ae8fdb214bf8e9a6","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-da30af73ae8fdb214bf8e9a6: start","job_status":"start","queue_duration_s":4.620914,"scheduling_latency_s":4.620938} gitlab # [ 185.848044] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.521Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"da30af73ae8fdb214bf8e9a6","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-da30af73ae8fdb214bf8e9a6: done: 0.015967 sec","job_status":"done","queue_duration_s":4.620914,"scheduling_latency_s":4.620938,"redis_calls":13,"redis_duration_s":0.003168,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001065,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002103,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010326,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015967,"completed_at":"2026-07-06T11:48:11.521Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 185.908576] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.523Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"36496d834375a572355ae621","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-36496d834375a572355ae621: start","job_status":"start","queue_duration_s":4.638827,"scheduling_latency_s":4.63885} gitlab # [ 185.986319] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.538Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"36496d834375a572355ae621","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-36496d834375a572355ae621: done: 0.015646 sec","job_status":"done","queue_duration_s":4.638827,"scheduling_latency_s":4.63885,"redis_calls":13,"redis_duration_s":0.003211,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000967,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002244,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010047,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015646,"completed_at":"2026-07-06T11:48:11.538Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 185.988157] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.540Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3ba73dead059fb4ecf38ada","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3ba73dead059fb4ecf38ada: start","job_status":"start","queue_duration_s":4.656346,"scheduling_latency_s":4.656367} gitlab # [ 186.005251] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.556Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3ba73dead059fb4ecf38ada","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3ba73dead059fb4ecf38ada: done: 0.015686 sec","job_status":"done","queue_duration_s":4.656346,"scheduling_latency_s":4.656367,"redis_calls":13,"redis_duration_s":0.002973,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010105,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015686,"completed_at":"2026-07-06T11:48:11.556Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 186.066263] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.558Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dcc9965170dfad2ee85c04ce","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dcc9965170dfad2ee85c04ce: start","job_status":"start","queue_duration_s":4.673628,"scheduling_latency_s":4.67365} gitlab # [ 186.085563] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.573Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dcc9965170dfad2ee85c04ce","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dcc9965170dfad2ee85c04ce: done: 0.015654 sec","job_status":"done","queue_duration_s":4.673628,"scheduling_latency_s":4.67365,"redis_calls":13,"redis_duration_s":0.00317,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001114,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002056,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009953,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015654,"completed_at":"2026-07-06T11:48:11.573Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 186.174188] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.575Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7caf4e6d3689bdfbe3b6866c","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7caf4e6d3689bdfbe3b6866c: start","job_status":"start","queue_duration_s":4.690692,"scheduling_latency_s":4.690714} gitlab # [ 186.175509] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.590Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7caf4e6d3689bdfbe3b6866c","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7caf4e6d3689bdfbe3b6866c: done: 0.015824 sec","job_status":"done","queue_duration_s":4.690692,"scheduling_latency_s":4.690714,"redis_calls":13,"redis_duration_s":0.003323,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000902,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002421,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009897,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015824,"completed_at":"2026-07-06T11:48:11.590Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 186.264140] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.592Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7243a635f426828ec690669d","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7243a635f426828ec690669d: start","job_status":"start","queue_duration_s":4.707836,"scheduling_latency_s":4.707859} gitlab # [ 186.265421] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.608Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7243a635f426828ec690669d","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7243a635f426828ec690669d: done: 0.016072 sec","job_status":"done","queue_duration_s":4.707836,"scheduling_latency_s":4.707859,"redis_calls":13,"redis_duration_s":0.003462,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000983,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002479,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010307,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016072,"completed_at":"2026-07-06T11:48:11.608Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 186.304986] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.609Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2bca080b35c78d3262f628e2","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2bca080b35c78d3262f628e2: start","job_status":"start","queue_duration_s":4.725325,"scheduling_latency_s":4.725347} gitlab # [ 186.324480] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.625Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2bca080b35c78d3262f628e2","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2bca080b35c78d3262f628e2: done: 0.015837 sec","job_status":"done","queue_duration_s":4.725325,"scheduling_latency_s":4.725347,"redis_calls":13,"redis_duration_s":0.0035780000000000005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001384,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002194,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010121,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015837,"completed_at":"2026-07-06T11:48:11.625Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 186.387294] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.626Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8aae86b24010510d50aaaf2c","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8aae86b24010510d50aaaf2c: start","job_status":"start","queue_duration_s":4.742458,"scheduling_latency_s":4.742479} gitlab # [ 186.406908] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.642Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8aae86b24010510d50aaaf2c","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8aae86b24010510d50aaaf2c: done: 0.015903 sec","job_status":"done","queue_duration_s":4.742458,"scheduling_latency_s":4.742479,"redis_calls":13,"redis_duration_s":0.0030310000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001007,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002024,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010373,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015903,"completed_at":"2026-07-06T11:48:11.642Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 186.469119] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.644Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0efba4b73eaf71c44d8d986f","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0efba4b73eaf71c44d8d986f: start","job_status":"start","queue_duration_s":4.759719,"scheduling_latency_s":4.759742} gitlab # [ 186.553404] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.660Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0efba4b73eaf71c44d8d986f","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0efba4b73eaf71c44d8d986f: done: 0.015839 sec","job_status":"done","queue_duration_s":4.759719,"scheduling_latency_s":4.759742,"redis_calls":13,"redis_duration_s":0.0030600000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000899,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002161,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010413,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015839,"completed_at":"2026-07-06T11:48:11.660Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 186.555070] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.661Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ad9770507db41c68cd62799","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ad9770507db41c68cd62799: start","job_status":"start","queue_duration_s":4.777122,"scheduling_latency_s":4.777145} gitlab # [ 186.568290] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.677Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ad9770507db41c68cd62799","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ad9770507db41c68cd62799: done: 0.015718 sec","job_status":"done","queue_duration_s":4.777122,"scheduling_latency_s":4.777145,"redis_calls":13,"redis_duration_s":0.003152,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000983,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002169,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009801,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015718,"completed_at":"2026-07-06T11:48:11.677Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 186.629936] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.678Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ab76a8d875316c0d9503204","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ab76a8d875316c0d9503204: start","job_status":"start","queue_duration_s":4.794341,"scheduling_latency_s":4.794363} gitlab # [ 186.649367] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.694Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ab76a8d875316c0d9503204","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ab76a8d875316c0d9503204: done: 0.015882 sec","job_status":"done","queue_duration_s":4.794341,"scheduling_latency_s":4.794363,"redis_calls":13,"redis_duration_s":0.003179,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000983,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002196,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010151,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015882,"completed_at":"2026-07-06T11:48:11.694Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 186.711695] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.696Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d1a8ebfc674be1c9a294446","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d1a8ebfc674be1c9a294446: start","job_status":"start","queue_duration_s":4.81167,"scheduling_latency_s":4.811692} gitlab # [ 186.730384] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.711Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d1a8ebfc674be1c9a294446","created_at":"2026-07-06T11:48:06.796Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-759a74f261014b11","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f80676893743015507f8b639a61359be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:06.884Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d1a8ebfc674be1c9a294446: done: 0.01559 sec","job_status":"done","queue_duration_s":4.81167,"scheduling_latency_s":4.811692,"redis_calls":13,"redis_duration_s":0.0031469999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001067,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00208,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009998,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01559,"completed_at":"2026-07-06T11:48:11.711Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 186.792317] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.713Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fe540ae70f1189d6c9695f7b","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fe540ae70f1189d6c9695f7b: start","job_status":"start","queue_duration_s":3.603804,"scheduling_latency_s":3.603827} gitlab # [ 186.811631] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.787Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fe540ae70f1189d6c9695f7b","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fe540ae70f1189d6c9695f7b: done: 0.074265 sec","job_status":"done","queue_duration_s":3.603804,"scheduling_latency_s":3.603827,"redis_calls":13,"redis_duration_s":0.003323,"redis_read_bytes":12,"redis_write_bytes":1300,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000529,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":289,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001289,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":395,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001505,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":616,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.006,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016991,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035647,"duration_s":0.074265,"completed_at":"2026-07-06T11:48:11.787Z","load_balancing_strategy":"primary","db_duration_s":0.008164,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 186.875394] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.789Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5b7e14628b0ccf0b02379ee8","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5b7e14628b0ccf0b02379ee8: start","job_status":"start","queue_duration_s":3.679855,"scheduling_latency_s":3.679885} gitlab # [ 186.894817] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5b7e14628b0ccf0b02379ee8","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5b7e14628b0ccf0b02379ee8: done: 0.018056 sec","job_status":"done","queue_duration_s":3.679855,"scheduling_latency_s":3.679885,"redis_calls":10,"redis_duration_s":0.002568,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001062,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001506,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01133,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018056,"completed_at":"2026-07-06T11:48:11.807Z","load_balancing_strategy":"primary","db_duration_s":0.001988,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 186.982330] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.808Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ff56bfceebb910197665a9ac","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ff56bfceebb910197665a9ac: start","job_status":"start","queue_duration_s":3.699273,"scheduling_latency_s":3.699296} gitlab # [ 186.983805] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.826Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ff56bfceebb910197665a9ac","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ff56bfceebb910197665a9ac: done: 0.017373 sec","job_status":"done","queue_duration_s":3.699273,"scheduling_latency_s":3.699296,"redis_calls":10,"redis_duration_s":0.002663,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001235,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001428,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010167,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017373,"completed_at":"2026-07-06T11:48:11.826Z","load_balancing_strategy":"primary","db_duration_s":0.001638,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 187.032270] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.827Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f0f3fffd2b95a21bfc143f17","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f0f3fffd2b95a21bfc143f17: start","job_status":"start","queue_duration_s":3.717965,"scheduling_latency_s":3.717998} gitlab # [ 187.051509] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.844Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f0f3fffd2b95a21bfc143f17","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f0f3fffd2b95a21bfc143f17: done: 0.016653 sec","job_status":"done","queue_duration_s":3.717965,"scheduling_latency_s":3.717998,"redis_calls":10,"redis_duration_s":0.002461,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000891,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00157,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010636,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016653,"completed_at":"2026-07-06T11:48:11.843Z","load_balancing_strategy":"primary","db_duration_s":0.001446,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 187.123084] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.845Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"bdbcb989c7f19bb018c693d7","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-bdbcb989c7f19bb018c693d7: start","job_status":"start","queue_duration_s":3.735915,"scheduling_latency_s":3.735938} gitlab # [ 187.128412] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.862Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"bdbcb989c7f19bb018c693d7","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-bdbcb989c7f19bb018c693d7: done: 0.016766 sec","job_status":"done","queue_duration_s":3.735915,"scheduling_latency_s":3.735938,"redis_calls":10,"redis_duration_s":0.002472,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00095,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001522,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01072,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016766,"completed_at":"2026-07-06T11:48:11.862Z","load_balancing_strategy":"primary","db_duration_s":0.001543,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 187.188827] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.863Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"62514c4d6c57862506579eea","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-62514c4d6c57862506579eea: start","job_status":"start","queue_duration_s":3.754251,"scheduling_latency_s":3.754276} gitlab # [ 187.261600] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.880Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"62514c4d6c57862506579eea","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-62514c4d6c57862506579eea: done: 0.017303 sec","job_status":"done","queue_duration_s":3.754251,"scheduling_latency_s":3.754276,"redis_calls":10,"redis_duration_s":0.002437,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000969,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001468,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010536,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017303,"completed_at":"2026-07-06T11:48:11.880Z","load_balancing_strategy":"primary","db_duration_s":0.001449,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 187.264333] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.882Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0d7bca803c93eb1e7cd1cb65","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0d7bca803c93eb1e7cd1cb65: start","job_status":"start","queue_duration_s":3.773509,"scheduling_latency_s":3.773532} gitlab # [ 187.283291] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.900Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0d7bca803c93eb1e7cd1cb65","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0d7bca803c93eb1e7cd1cb65: done: 0.017795 sec","job_status":"done","queue_duration_s":3.773509,"scheduling_latency_s":3.773532,"redis_calls":10,"redis_duration_s":0.002453,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000973,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00148,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011341,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017795,"completed_at":"2026-07-06T11:48:11.900Z","load_balancing_strategy":"primary","db_duration_s":0.001532,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 187.343113] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.902Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a74cab4dc3f19efdc7795987","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a74cab4dc3f19efdc7795987: start","job_status":"start","queue_duration_s":3.792691,"scheduling_latency_s":3.792714} gitlab # [ 187.361846] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.919Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a74cab4dc3f19efdc7795987","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a74cab4dc3f19efdc7795987: done: 0.017881 sec","job_status":"done","queue_duration_s":3.792691,"scheduling_latency_s":3.792714,"redis_calls":10,"redis_duration_s":0.0026839999999999998,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000986,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001698,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011365,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017881,"completed_at":"2026-07-06T11:48:11.919Z","load_balancing_strategy":"primary","db_duration_s":0.002077,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 187.422520] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.921Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"db036233890cab8a254bbc0f","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-db036233890cab8a254bbc0f: start","job_status":"start","queue_duration_s":3.812058,"scheduling_latency_s":3.812124} gitlab # [ 187.441915] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.939Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"db036233890cab8a254bbc0f","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-db036233890cab8a254bbc0f: done: 0.017833 sec","job_status":"done","queue_duration_s":3.812058,"scheduling_latency_s":3.812124,"redis_calls":10,"redis_duration_s":0.002464,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00091,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001554,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011069,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017833,"completed_at":"2026-07-06T11:48:11.939Z","load_balancing_strategy":"primary","db_duration_s":0.001682,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 187.522242] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.940Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f2f3e6a7cfd9a813c84132b9","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f2f3e6a7cfd9a813c84132b9: start","job_status":"start","queue_duration_s":3.831419,"scheduling_latency_s":3.831442} gitlab # [ 187.523510] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.957Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f2f3e6a7cfd9a813c84132b9","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f2f3e6a7cfd9a813c84132b9: done: 0.017173 sec","job_status":"done","queue_duration_s":3.831419,"scheduling_latency_s":3.831442,"redis_calls":10,"redis_duration_s":0.002313,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000898,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001415,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010375,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017173,"completed_at":"2026-07-06T11:48:11.957Z","load_balancing_strategy":"primary","db_duration_s":0.001518,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 187.579432] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.960Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c8afcc5d8df59d94ee3d98c8","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c8afcc5d8df59d94ee3d98c8: start","job_status":"start","queue_duration_s":3.850641,"scheduling_latency_s":3.850662} gitlab # [ 187.599071] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.977Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c8afcc5d8df59d94ee3d98c8","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c8afcc5d8df59d94ee3d98c8: done: 0.017134 sec","job_status":"done","queue_duration_s":3.850641,"scheduling_latency_s":3.850662,"redis_calls":10,"redis_duration_s":0.00238,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000972,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001408,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010775,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017134,"completed_at":"2026-07-06T11:48:11.977Z","load_balancing_strategy":"primary","db_duration_s":0.001375,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 187.687229] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.978Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"78d2ffe1037acbffd16a9f9f","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-78d2ffe1037acbffd16a9f9f: start","job_status":"start","queue_duration_s":3.869188,"scheduling_latency_s":3.86921} gitlab # [ 187.688500] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.994Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"78d2ffe1037acbffd16a9f9f","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-78d2ffe1037acbffd16a9f9f: done: 0.016282 sec","job_status":"done","queue_duration_s":3.869188,"scheduling_latency_s":3.86921,"redis_calls":10,"redis_duration_s":0.002427,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000896,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001531,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0104,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016282,"completed_at":"2026-07-06T11:48:11.994Z","load_balancing_strategy":"primary","db_duration_s":0.001318,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 187.737511] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:11.996Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"af23f33ef1cfed7ebc2aa17e","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-af23f33ef1cfed7ebc2aa17e: start","job_status":"start","queue_duration_s":3.886792,"scheduling_latency_s":3.886815} gitlab # [ 187.756524] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.012Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"af23f33ef1cfed7ebc2aa17e","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-af23f33ef1cfed7ebc2aa17e: done: 0.016468 sec","job_status":"done","queue_duration_s":3.886792,"scheduling_latency_s":3.886815,"redis_calls":10,"redis_duration_s":0.00243,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000983,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001447,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010564,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016468,"completed_at":"2026-07-06T11:48:12.012Z","load_balancing_strategy":"primary","db_duration_s":0.00148,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 187.817323] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.014Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ee762910e4543c91b77f03b5","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ee762910e4543c91b77f03b5: start","job_status":"start","queue_duration_s":3.904551,"scheduling_latency_s":3.904573} gitlab # [ 187.836912] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.030Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ee762910e4543c91b77f03b5","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ee762910e4543c91b77f03b5: done: 0.016622 sec","job_status":"done","queue_duration_s":3.904551,"scheduling_latency_s":3.904573,"redis_calls":10,"redis_duration_s":0.002875,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00095,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001925,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009903,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016622,"completed_at":"2026-07-06T11:48:12.030Z","load_balancing_strategy":"primary","db_duration_s":0.001364,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 187.897171] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.032Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6bec2651bf2ffbbbcd050fce","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6bec2651bf2ffbbbcd050fce: start","job_status":"start","queue_duration_s":3.923001,"scheduling_latency_s":3.923023} gitlab # [ 187.971317] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.048Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6bec2651bf2ffbbbcd050fce","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6bec2651bf2ffbbbcd050fce: done: 0.016305 sec","job_status":"done","queue_duration_s":3.923001,"scheduling_latency_s":3.923023,"redis_calls":10,"redis_duration_s":0.002427,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000872,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001555,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010534,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016305,"completed_at":"2026-07-06T11:48:12.048Z","load_balancing_strategy":"primary","db_duration_s":0.001419,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 187.974531] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.050Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b410a9ff497db38afd4741e4","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b410a9ff497db38afd4741e4: start","job_status":"start","queue_duration_s":3.940615,"scheduling_latency_s":3.940636} gitlab # [ 187.994187] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.066Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b410a9ff497db38afd4741e4","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b410a9ff497db38afd4741e4: done: 0.016745 sec","job_status":"done","queue_duration_s":3.940615,"scheduling_latency_s":3.940636,"redis_calls":10,"redis_duration_s":0.003151,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000977,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002174,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010787,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016745,"completed_at":"2026-07-06T11:48:12.066Z","load_balancing_strategy":"primary","db_duration_s":0.001511,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 188.112580] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.068Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d7d6beef5e1fa3a1f9299910","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d7d6beef5e1fa3a1f9299910: start","job_status":"start","queue_duration_s":3.958635,"scheduling_latency_s":3.958656} gitlab # [ 188.114260] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.084Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d7d6beef5e1fa3a1f9299910","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d7d6beef5e1fa3a1f9299910: done: 0.016476 sec","job_status":"done","queue_duration_s":3.958635,"scheduling_latency_s":3.958656,"redis_calls":10,"redis_duration_s":0.002312,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000957,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001355,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010681,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016476,"completed_at":"2026-07-06T11:48:12.084Z","load_balancing_strategy":"primary","db_duration_s":0.001359,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 188.132101] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.085Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"18b9c1963b6328892dd9e6e1","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-18b9c1963b6328892dd9e6e1: start","job_status":"start","queue_duration_s":3.976464,"scheduling_latency_s":3.976485} gitlab # [ 188.151254] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.102Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"18b9c1963b6328892dd9e6e1","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-18b9c1963b6328892dd9e6e1: done: 0.016857 sec","job_status":"done","queue_duration_s":3.976464,"scheduling_latency_s":3.976485,"redis_calls":10,"redis_duration_s":0.002418,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000923,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001495,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00969,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016857,"completed_at":"2026-07-06T11:48:12.102Z","load_balancing_strategy":"primary","db_duration_s":0.001412,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 188.254339] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.104Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d0e1fb3cbb0643be9dc7ce8d","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d0e1fb3cbb0643be9dc7ce8d: start","job_status":"start","queue_duration_s":3.994665,"scheduling_latency_s":3.994686} gitlab # [ 188.255862] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.121Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d0e1fb3cbb0643be9dc7ce8d","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d0e1fb3cbb0643be9dc7ce8d: done: 0.017332 sec","job_status":"done","queue_duration_s":3.994665,"scheduling_latency_s":3.994686,"redis_calls":10,"redis_duration_s":0.002758,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001163,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001595,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010583,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017332,"completed_at":"2026-07-06T11:48:12.121Z","load_balancing_strategy":"primary","db_duration_s":0.002379,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 188.289443] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.122Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b9766597376045d63b54dea4","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b9766597376045d63b54dea4: start","job_status":"start","queue_duration_s":4.013545,"scheduling_latency_s":4.013567} gitlab # [ 188.308876] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.148Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b9766597376045d63b54dea4","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b9766597376045d63b54dea4: done: 0.025261 sec","job_status":"done","queue_duration_s":4.013545,"scheduling_latency_s":4.013567,"redis_calls":10,"redis_duration_s":0.004459,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002118,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002341,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017077,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025261,"completed_at":"2026-07-06T11:48:12.148Z","load_balancing_strategy":"primary","db_duration_s":0.0019,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 188.396182] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.149Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ec2dd00bc84dedeba0167243","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ec2dd00bc84dedeba0167243: start","job_status":"start","queue_duration_s":4.040374,"scheduling_latency_s":4.040401} gitlab # [ 188.397575] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.167Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ec2dd00bc84dedeba0167243","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ec2dd00bc84dedeba0167243: done: 0.017293 sec","job_status":"done","queue_duration_s":4.040374,"scheduling_latency_s":4.040401,"redis_calls":10,"redis_duration_s":0.002397,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001007,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00139,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010844,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017293,"completed_at":"2026-07-06T11:48:12.167Z","load_balancing_strategy":"primary","db_duration_s":0.001597,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 188.446880] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.168Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a18905af25075f524bfbf4f2","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a18905af25075f524bfbf4f2: start","job_status":"start","queue_duration_s":4.058955,"scheduling_latency_s":4.059489} gitlab # [ 188.466047] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.186Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a18905af25075f524bfbf4f2","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a18905af25075f524bfbf4f2: done: 0.018294 sec","job_status":"done","queue_duration_s":4.058955,"scheduling_latency_s":4.059489,"redis_calls":10,"redis_duration_s":0.002508,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000897,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001611,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018294,"completed_at":"2026-07-06T11:48:12.186Z","load_balancing_strategy":"primary","db_duration_s":0.001322,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 188.537157] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.188Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f240988a6ec2cd49e8fa8db8","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f240988a6ec2cd49e8fa8db8: start","job_status":"start","queue_duration_s":4.078573,"scheduling_latency_s":4.078595} gitlab # [ 188.542908] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.204Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f240988a6ec2cd49e8fa8db8","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f240988a6ec2cd49e8fa8db8: done: 0.016408 sec","job_status":"done","queue_duration_s":4.078573,"scheduling_latency_s":4.078595,"redis_calls":10,"redis_duration_s":0.002417,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001033,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001384,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010626,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016408,"completed_at":"2026-07-06T11:48:12.204Z","load_balancing_strategy":"primary","db_duration_s":0.001378,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 188.603409] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.205Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9be0f01f6e4dd3c8ae35db0c","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9be0f01f6e4dd3c8ae35db0c: start","job_status":"start","queue_duration_s":4.096339,"scheduling_latency_s":4.096359} gitlab # [ 188.622967] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.221Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9be0f01f6e4dd3c8ae35db0c","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9be0f01f6e4dd3c8ae35db0c: done: 0.016217 sec","job_status":"done","queue_duration_s":4.096339,"scheduling_latency_s":4.096359,"redis_calls":10,"redis_duration_s":0.002392,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000898,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001494,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010501,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016217,"completed_at":"2026-07-06T11:48:12.221Z","load_balancing_strategy":"primary","db_duration_s":0.001429,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 188.716333] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.223Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5141c698a0e47a372ff09422","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5141c698a0e47a372ff09422: start","job_status":"start","queue_duration_s":4.113885,"scheduling_latency_s":4.113907} gitlab # [ 188.718068] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.239Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5141c698a0e47a372ff09422","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5141c698a0e47a372ff09422: done: 0.016676 sec","job_status":"done","queue_duration_s":4.113885,"scheduling_latency_s":4.113907,"redis_calls":10,"redis_duration_s":0.002466,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000952,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001514,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010006,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016676,"completed_at":"2026-07-06T11:48:12.239Z","load_balancing_strategy":"primary","db_duration_s":0.001314,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 188.811716] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.241Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"21a21576020b5728e8fe5d97","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-21a21576020b5728e8fe5d97: start","job_status":"start","queue_duration_s":4.132454,"scheduling_latency_s":4.132475} gitlab # [ 188.813151] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.261Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"21a21576020b5728e8fe5d97","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-21a21576020b5728e8fe5d97: done: 0.019307 sec","job_status":"done","queue_duration_s":4.132454,"scheduling_latency_s":4.132475,"redis_calls":10,"redis_duration_s":0.002581,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000951,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00163,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013314,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019307,"completed_at":"2026-07-06T11:48:12.261Z","load_balancing_strategy":"primary","db_duration_s":0.001435,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 188.836845] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.262Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"2f6fd75901b566d93bdf20e9","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-2f6fd75901b566d93bdf20e9: start","job_status":"start","queue_duration_s":4.153068,"scheduling_latency_s":4.153131} gitlab # [ 188.856232] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.278Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"2f6fd75901b566d93bdf20e9","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-2f6fd75901b566d93bdf20e9: done: 0.016278 sec","job_status":"done","queue_duration_s":4.153068,"scheduling_latency_s":4.153131,"redis_calls":10,"redis_duration_s":0.00251,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000887,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001623,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010366,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016278,"completed_at":"2026-07-06T11:48:12.278Z","load_balancing_strategy":"primary","db_duration_s":0.001432,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 188.953756] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.280Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"06d925236d4a70fa3d94c15b","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-06d925236d4a70fa3d94c15b: start","job_status":"start","queue_duration_s":4.170676,"scheduling_latency_s":4.170698} gitlab # [ 188.958199] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.296Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"06d925236d4a70fa3d94c15b","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-06d925236d4a70fa3d94c15b: done: 0.01607 sec","job_status":"done","queue_duration_s":4.170676,"scheduling_latency_s":4.170698,"redis_calls":10,"redis_duration_s":0.002409,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000984,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001425,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01607,"completed_at":"2026-07-06T11:48:12.296Z","load_balancing_strategy":"primary","db_duration_s":0.001375,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 188.996349] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.297Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8caace23be17b1e96e0bbd8d","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8caace23be17b1e96e0bbd8d: start","job_status":"start","queue_duration_s":4.188056,"scheduling_latency_s":4.188117} gitlab # [ 189.015293] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.313Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8caace23be17b1e96e0bbd8d","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8caace23be17b1e96e0bbd8d: done: 0.016124 sec","job_status":"done","queue_duration_s":4.188056,"scheduling_latency_s":4.188117,"redis_calls":10,"redis_duration_s":0.0024070000000000003,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000987,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00142,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009904,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016124,"completed_at":"2026-07-06T11:48:12.313Z","load_balancing_strategy":"primary","db_duration_s":0.001411,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 189.102230] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.315Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6e7bdde4b2cb03c29a8eb1f4","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6e7bdde4b2cb03c29a8eb1f4: start","job_status":"start","queue_duration_s":4.205867,"scheduling_latency_s":4.205889} gitlab # [ 189.103514] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.332Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6e7bdde4b2cb03c29a8eb1f4","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6e7bdde4b2cb03c29a8eb1f4: done: 0.016932 sec","job_status":"done","queue_duration_s":4.205867,"scheduling_latency_s":4.205889,"redis_calls":10,"redis_duration_s":0.002582,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000929,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001653,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010608,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016932,"completed_at":"2026-07-06T11:48:12.332Z","load_balancing_strategy":"primary","db_duration_s":0.001393,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 189.153271] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.333Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5cda77bdd38a8238eca0a0a3","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5cda77bdd38a8238eca0a0a3: start","job_status":"start","queue_duration_s":4.224165,"scheduling_latency_s":4.224187} gitlab # [ 189.172326] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.350Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5cda77bdd38a8238eca0a0a3","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5cda77bdd38a8238eca0a0a3: done: 0.016925 sec","job_status":"done","queue_duration_s":4.224165,"scheduling_latency_s":4.224187,"redis_calls":10,"redis_duration_s":0.00228,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000936,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001344,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016925,"completed_at":"2026-07-06T11:48:12.350Z","load_balancing_strategy":"primary","db_duration_s":0.001491,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 189.242930] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.351Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c6db06e303cf9cbaadb52a46","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c6db06e303cf9cbaadb52a46: start","job_status":"start","queue_duration_s":4.242413,"scheduling_latency_s":4.242434} gitlab # [ 189.249089] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.367Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c6db06e303cf9cbaadb52a46","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c6db06e303cf9cbaadb52a46: done: 0.016117 sec","job_status":"done","queue_duration_s":4.242413,"scheduling_latency_s":4.242434,"redis_calls":10,"redis_duration_s":0.002375,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001003,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001372,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009912,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016117,"completed_at":"2026-07-06T11:48:12.367Z","load_balancing_strategy":"primary","db_duration_s":0.001523,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 189.309251] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.369Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"05ad524b0e9016189d5c1f1b","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-05ad524b0e9016189d5c1f1b: start","job_status":"start","queue_duration_s":4.259834,"scheduling_latency_s":4.259854} gitlab # [ 189.383661] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.385Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"05ad524b0e9016189d5c1f1b","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-05ad524b0e9016189d5c1f1b: done: 0.016466 sec","job_status":"done","queue_duration_s":4.259834,"scheduling_latency_s":4.259854,"redis_calls":10,"redis_duration_s":0.002345,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000956,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001389,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010676,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016466,"completed_at":"2026-07-06T11:48:12.385Z","load_balancing_strategy":"primary","db_duration_s":0.001553,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 189.386648] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.387Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d32b2acea2af72973c8c5e1e","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d32b2acea2af72973c8c5e1e: start","job_status":"start","queue_duration_s":4.277606,"scheduling_latency_s":4.277627} gitlab # [ 189.407155] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.404Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d32b2acea2af72973c8c5e1e","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d32b2acea2af72973c8c5e1e: done: 0.017426 sec","job_status":"done","queue_duration_s":4.277606,"scheduling_latency_s":4.277627,"redis_calls":10,"redis_duration_s":0.00244,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001078,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001362,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01068,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017426,"completed_at":"2026-07-06T11:48:12.404Z","load_balancing_strategy":"primary","db_duration_s":0.001457,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 189.524986] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.405Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7889512d82f23504b78e7805","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7889512d82f23504b78e7805: start","job_status":"start","queue_duration_s":4.296406,"scheduling_latency_s":4.296429} gitlab # [ 189.526588] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.422Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7889512d82f23504b78e7805","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7889512d82f23504b78e7805: done: 0.016772 sec","job_status":"done","queue_duration_s":4.296406,"scheduling_latency_s":4.296429,"redis_calls":10,"redis_duration_s":0.002394,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001001,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001393,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010898,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016772,"completed_at":"2026-07-06T11:48:12.422Z","load_balancing_strategy":"primary","db_duration_s":0.001381,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 189.545498] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.424Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3a5722ac96db16bc4195216d","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3a5722ac96db16bc4195216d: start","job_status":"start","queue_duration_s":4.314501,"scheduling_latency_s":4.315295} gitlab # [ 189.565268] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.441Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3a5722ac96db16bc4195216d","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3a5722ac96db16bc4195216d: done: 0.017582 sec","job_status":"done","queue_duration_s":4.314501,"scheduling_latency_s":4.315295,"redis_calls":10,"redis_duration_s":0.0028009999999999997,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001127,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001674,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010665,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017582,"completed_at":"2026-07-06T11:48:12.441Z","load_balancing_strategy":"primary","db_duration_s":0.001312,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 189.666966] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.442Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e24e4dc46a051e390121522f","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e24e4dc46a051e390121522f: start","job_status":"start","queue_duration_s":4.333443,"scheduling_latency_s":4.333464} gitlab # [ 189.668895] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.461Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e24e4dc46a051e390121522f","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e24e4dc46a051e390121522f: done: 0.018338 sec","job_status":"done","queue_duration_s":4.333443,"scheduling_latency_s":4.333464,"redis_calls":10,"redis_duration_s":0.002419,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001015,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001404,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010782,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018338,"completed_at":"2026-07-06T11:48:12.461Z","load_balancing_strategy":"primary","db_duration_s":0.001362,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 189.703965] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.463Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3ed05cf89de32e1521206be4","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3ed05cf89de32e1521206be4: start","job_status":"start","queue_duration_s":4.353925,"scheduling_latency_s":4.353946} gitlab # [ 189.722352] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.480Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3ed05cf89de32e1521206be4","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3ed05cf89de32e1521206be4: done: 0.016869 sec","job_status":"done","queue_duration_s":4.353925,"scheduling_latency_s":4.353946,"redis_calls":10,"redis_duration_s":0.0025340000000000002,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000952,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001582,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010846,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016869,"completed_at":"2026-07-06T11:48:12.480Z","load_balancing_strategy":"primary","db_duration_s":0.001497,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 189.782245] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.481Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"92555b95f46c7ecf957623f5","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-92555b95f46c7ecf957623f5: start","job_status":"start","queue_duration_s":4.372176,"scheduling_latency_s":4.372198} gitlab # [ 189.801124] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.500Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"92555b95f46c7ecf957623f5","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-92555b95f46c7ecf957623f5: done: 0.01842 sec","job_status":"done","queue_duration_s":4.372176,"scheduling_latency_s":4.372198,"redis_calls":10,"redis_duration_s":0.002602,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000896,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001706,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011968,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01842,"completed_at":"2026-07-06T11:48:12.499Z","load_balancing_strategy":"primary","db_duration_s":0.00206,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 189.860662] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.501Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4b33d2d7017f3223aa319da3","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4b33d2d7017f3223aa319da3: start","job_status":"start","queue_duration_s":4.391961,"scheduling_latency_s":4.391995} gitlab # [ 189.941383] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.518Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4b33d2d7017f3223aa319da3","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4b33d2d7017f3223aa319da3: done: 0.017105 sec","job_status":"done","queue_duration_s":4.391961,"scheduling_latency_s":4.391995,"redis_calls":10,"redis_duration_s":0.002814,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001909,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011091,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017105,"completed_at":"2026-07-06T11:48:12.518Z","load_balancing_strategy":"primary","db_duration_s":0.001413,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 189.943255] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.519Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4d3cf7775fdec6f4c619fa53","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4d3cf7775fdec6f4c619fa53: start","job_status":"start","queue_duration_s":4.410409,"scheduling_latency_s":4.410431} gitlab # [ 189.956212] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.538Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4d3cf7775fdec6f4c619fa53","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4d3cf7775fdec6f4c619fa53: done: 0.019001 sec","job_status":"done","queue_duration_s":4.410409,"scheduling_latency_s":4.410431,"redis_calls":10,"redis_duration_s":0.002706,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001029,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001677,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010118,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019001,"completed_at":"2026-07-06T11:48:12.538Z","load_balancing_strategy":"primary","db_duration_s":0.001517,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 190.017226] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.540Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"39aa1d04085b6c7f215df5a3","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-39aa1d04085b6c7f215df5a3: start","job_status":"start","queue_duration_s":4.430742,"scheduling_latency_s":4.430763} gitlab # [ 190.036439] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.733Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"39aa1d04085b6c7f215df5a3","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-39aa1d04085b6c7f215df5a3: done: 0.193556 sec","job_status":"done","queue_duration_s":4.430742,"scheduling_latency_s":4.430763,"redis_calls":10,"redis_duration_s":0.002788,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001319,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001469,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012588,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.193556,"completed_at":"2026-07-06T11:48:12.733Z","load_balancing_strategy":"primary","db_duration_s":0.001376,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 190.096996] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.735Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a062cc0b9803ac2ee9383706","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a062cc0b9803ac2ee9383706: start","job_status":"start","queue_duration_s":4.625964,"scheduling_latency_s":4.626011} gitlab # [ 190.116410] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.755Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a062cc0b9803ac2ee9383706","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a062cc0b9803ac2ee9383706: done: 0.020351 sec","job_status":"done","queue_duration_s":4.625964,"scheduling_latency_s":4.626011,"redis_calls":10,"redis_duration_s":0.002516,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000985,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001531,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012443,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020351,"completed_at":"2026-07-06T11:48:12.755Z","load_balancing_strategy":"primary","db_duration_s":0.002082,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 190.205199] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.757Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c9ae968846c41419f4f0d0ec","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c9ae968846c41419f4f0d0ec: start","job_status":"start","queue_duration_s":4.647706,"scheduling_latency_s":4.647729} gitlab # [ 190.206605] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.773Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c9ae968846c41419f4f0d0ec","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c9ae968846c41419f4f0d0ec: done: 0.016635 sec","job_status":"done","queue_duration_s":4.647706,"scheduling_latency_s":4.647729,"redis_calls":10,"redis_duration_s":0.002422,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000897,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001525,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010571,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016635,"completed_at":"2026-07-06T11:48:12.773Z","load_balancing_strategy":"primary","db_duration_s":0.001435,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 190.254306] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.775Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c8a05396773a4f0129615faf","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c8a05396773a4f0129615faf: start","job_status":"start","queue_duration_s":4.665629,"scheduling_latency_s":4.665651} gitlab # [ 190.273195] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.791Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c8a05396773a4f0129615faf","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c8a05396773a4f0129615faf: done: 0.016192 sec","job_status":"done","queue_duration_s":4.665629,"scheduling_latency_s":4.665651,"redis_calls":10,"redis_duration_s":0.00277,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001825,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010252,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016192,"completed_at":"2026-07-06T11:48:12.791Z","load_balancing_strategy":"primary","db_duration_s":0.001501,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 190.333376] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.792Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ed7fd6c6ea86cbbb3a6e7b3d","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ed7fd6c6ea86cbbb3a6e7b3d: start","job_status":"start","queue_duration_s":4.683195,"scheduling_latency_s":4.683215} gitlab # [ 190.352582] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.811Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ed7fd6c6ea86cbbb3a6e7b3d","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ed7fd6c6ea86cbbb3a6e7b3d: done: 0.01863 sec","job_status":"done","queue_duration_s":4.683195,"scheduling_latency_s":4.683215,"redis_calls":10,"redis_duration_s":0.002373,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000916,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001457,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011259,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01863,"completed_at":"2026-07-06T11:48:12.811Z","load_balancing_strategy":"primary","db_duration_s":0.001431,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 190.413137] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.812Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3f0ef2fb6cdfb25aecf93abd","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3f0ef2fb6cdfb25aecf93abd: start","job_status":"start","queue_duration_s":4.703209,"scheduling_latency_s":4.703231} gitlab # [ 190.432153] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.828Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3f0ef2fb6cdfb25aecf93abd","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3f0ef2fb6cdfb25aecf93abd: done: 0.015811 sec","job_status":"done","queue_duration_s":4.703209,"scheduling_latency_s":4.703231,"redis_calls":10,"redis_duration_s":0.002258,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000913,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001345,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009908,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015811,"completed_at":"2026-07-06T11:48:12.828Z","load_balancing_strategy":"primary","db_duration_s":0.001367,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 190.518567] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.829Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"df758e4eedeb2efd2bd28eba","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-df758e4eedeb2efd2bd28eba: start","job_status":"start","queue_duration_s":4.720432,"scheduling_latency_s":4.720454} gitlab # [ 190.520606] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.847Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"df758e4eedeb2efd2bd28eba","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-df758e4eedeb2efd2bd28eba: done: 0.017473 sec","job_status":"done","queue_duration_s":4.720432,"scheduling_latency_s":4.720454,"redis_calls":10,"redis_duration_s":0.0024939999999999997,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000972,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001522,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010888,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017473,"completed_at":"2026-07-06T11:48:12.847Z","load_balancing_strategy":"primary","db_duration_s":0.001669,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 190.571136] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.848Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4b9e7b88a3f6f358a7295d49","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4b9e7b88a3f6f358a7295d49: start","job_status":"start","queue_duration_s":4.739294,"scheduling_latency_s":4.739316} gitlab # [ 190.590381] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.864Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4b9e7b88a3f6f358a7295d49","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4b9e7b88a3f6f358a7295d49: done: 0.016203 sec","job_status":"done","queue_duration_s":4.739294,"scheduling_latency_s":4.739316,"redis_calls":10,"redis_duration_s":0.0023179999999999997,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000946,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001372,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010065,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016203,"completed_at":"2026-07-06T11:48:12.864Z","load_balancing_strategy":"primary","db_duration_s":0.001488,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 190.660775] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.866Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4a7c62b1d31a622cedce9d32","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4a7c62b1d31a622cedce9d32: start","job_status":"start","queue_duration_s":4.756793,"scheduling_latency_s":4.756815} gitlab # [ 190.668647] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:12.882Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4a7c62b1d31a622cedce9d32","created_at":"2026-07-06T11:48:08.043Z","trace_propagation_headers":{"sentry-trace":"a817a112ea3e4f1eb992411bbe0edb33-91ec0a0d5fc7449c","baggage":"sentry-trace_id=a817a112ea3e4f1eb992411bbe0edb33,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"ee0b1767220ec4f3713567f5858aee81","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:08.109Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4a7c62b1d31a622cedce9d32: done: 0.016117 sec","job_status":"done","queue_duration_s":4.756793,"scheduling_latency_s":4.756815,"redis_calls":10,"redis_duration_s":0.002345,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000925,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00142,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01021,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016117,"completed_at":"2026-07-06T11:48:12.882Z","load_balancing_strategy":"primary","db_duration_s":0.001361,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 204.051185] runuser[2779]: pam_unix(runuser:session): session closed for user gitlab gitlab # [ 204.058462] setup-gitlab-start[3013]: ++ jq -r .token gitlab # [ 204.061456] setup-gitlab-start[3014]: +++ date -d '+7 days' +%F gitlab # [ 204.064769] setup-gitlab-start[3012]: ++ curl -fs -X POST -H 'Authorization: Bearer d671e376603561162e9671344be06509817bf784d0b1056733bc6ad47e430cfa' -H 'Content-Type: application/json' -d '{"name":"nixbot","scopes":["api"],"expires_at":"2026-07-13"}' http://gitlab/api/v4/users/1/personal_access_tokens gitlab # [ 204.680388] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:30.635Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"a0b25eb74b94a47af13da90a","created_at":"2026-07-06T11:48:30.603Z","trace_propagation_headers":{"sentry-trace":"b67a4b8399ea42deb141606ceaa81823-100494a126354fb2","baggage":"sentry-trace_id=b67a4b8399ea42deb141606ceaa81823,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM49PMN97G67VFG85SF0KV","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:40b2e6af08479f51c91e7033a4463da97b450de5a07f74cf2e043b05638542d6","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:30.630Z","job_size_bytes":451,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-a0b25eb74b94a47af13da90a: start","job_status":"start","queue_duration_s":0.005177,"scheduling_latency_s":0.005223} gitlab # [ 204.736725] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:30 +0000] "POST /api/v4/users/1/personal_access_tokens HTTP/1.1" 201 267 "" "curl/8.20.0" 614 gitlab # [ 204.740308] setup-gitlab-start[2737]: + TOKEN=glpat-e1OeptbxmaRucTIydj-R9G86MQp1OjEH.01.0w1gnfhwj gitlab # [ 204.742313] setup-gitlab-start[2737]: + test -n glpat-e1OeptbxmaRucTIydj-R9G86MQp1OjEH.01.0w1gnfhwj gitlab # [ 204.744324] setup-gitlab-start[2737]: + test glpat-e1OeptbxmaRucTIydj-R9G86MQp1OjEH.01.0w1gnfhwj '!=' null gitlab # [ 204.746453] setup-gitlab-start[2737]: + mkdir -p /var/lib/secrets gitlab # [ 204.747874] setup-gitlab-start[2737]: + echo glpat-e1OeptbxmaRucTIydj-R9G86MQp1OjEH.01.0w1gnfhwj gitlab # [ 204.749865] setup-gitlab-start[2737]: + chmod 644 /var/lib/secrets/gitlab-token gitlab # [ 204.751681] setup-gitlab-start[2737]: + curl -fs -X POST -H 'PRIVATE-TOKEN: glpat-e1OeptbxmaRucTIydj-R9G86MQp1OjEH.01.0w1gnfhwj' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects gitlab # [ 204.803554] sidekiq-cluster[2163]: {"severity":"WARN","time":"2026-07-06T11:48:30.759Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"a0b25eb74b94a47af13da90a","created_at":"2026-07-06T11:48:30.603Z","trace_propagation_headers":{"sentry-trace":"b67a4b8399ea42deb141606ceaa81823-100494a126354fb2","baggage":"sentry-trace_id=b67a4b8399ea42deb141606ceaa81823,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM49PMN97G67VFG85SF0KV","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:40b2e6af08479f51c91e7033a4463da97b450de5a07f74cf2e043b05638542d6","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:30.630Z","job_size_bytes":451,"pid":2163,"sidekiq_tid":"lymb","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-a0b25eb74b94a47af13da90a: fail: 0.123571 sec","job_status":"fail","queue_duration_s":0.005177,"scheduling_latency_s":0.005223,"redis_calls":11,"redis_duration_s":0.003377,"redis_read_bytes":137,"redis_write_bytes":1317,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000778,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001865,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000734,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.032,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.06573,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000413461,"duration_s":0.123571,"completed_at":"2026-07-06T11:48:30.758Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.045123,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 206.761423] gitaly[1389]: time="2026-07-06T11:48:32.716Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4ABM7PWJPMH6E51SNF91 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T11:48:42.004" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:32.715" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.765524] gitaly[1389]: time="2026-07-06T11:48:32.716Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4ABM7PWJPMH6E51SNF91 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T11:48:42.004" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:32.715" grpc.time_ms=0.948 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.938434] gitaly[1389]: time="2026-07-06T11:48:32.894Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4ABM7PWJPMH6E51SNF91 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T11:48:42.183" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:32.894" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.941323] gitaly[1389]: time="2026-07-06T11:48:32.895Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4ABM7PWJPMH6E51SNF91 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T11:48:42.183" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:32.894" grpc.time_ms=0.633 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.984509] gitaly[1389]: time="2026-07-06T11:48:32.940Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4ABM7PWJPMH6E51SNF91 diskcache=4a83e5ff-5842-488a-9e87-98fef376b077 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-06T11:48:42.191" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:32.902" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.987926] gitaly[1389]: time="2026-07-06T11:48:32.941Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=321 command.oublock=32 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4ABM7PWJPMH6E51SNF91 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-06T11:48:42.191" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:32.902" grpc.time_ms=38.521 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.007174] gitaly[1389]: time="2026-07-06T11:48:32.963Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4ABM7PWJPMH6E51SNF91 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T11:48:42.252" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:32.962" pid=1389 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 207.008798] gitaly[1389]: time="2026-07-06T11:48:32.963Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4ABM7PWJPMH6E51SNF91 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T11:48:42.252" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:32.962" grpc.time_ms=0.52 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.027945] gitaly[1389]: time="2026-07-06T11:48:32.984Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4ABM7PWJPMH6E51SNF91 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-06T11:48:42.258" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:32.969" grpc.time_ms=13.922 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.060951] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:33.015Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"997d4bc4076daaa94a1e63ec","created_at":"2026-07-06T11:48:32.998Z","trace_propagation_headers":{"sentry-trace":"3a333f0d213849c78e500f013ad17203-7a6a7a2d3b6a414a","baggage":"sentry-trace_id=3a333f0d213849c78e500f013ad17203,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4ABM7PWJPMH6E51SNF91","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:33.013Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-997d4bc4076daaa94a1e63ec: start","job_status":"start","queue_duration_s":0.002199,"scheduling_latency_s":0.002246} gitlab # [ 207.081090] gitaly[1389]: time="2026-07-06T11:48:33.037Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4ABM7PWJPMH6E51SNF91 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T11:48:43.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:33.037" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.082531] gitaly[1389]: time="2026-07-06T11:48:33.037Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4ABM7PWJPMH6E51SNF91 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T11:48:43.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:33.037" grpc.time_ms=0.261 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.117836] gitaly[1389]: time="2026-07-06T11:48:33.074Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4ABM7PWJPMH6E51SNF91 diskcache=5f86df54-d64c-437b-b82b-2a5c986dafc8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-06T11:48:43.010" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:33.041" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.119458] gitaly[1389]: time="2026-07-06T11:48:33.074Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=32 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4ABM7PWJPMH6E51SNF91 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-06T11:48:43.010" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:33.041" grpc.time_ms=32.829 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.130708] gitaly[1389]: time="2026-07-06T11:48:33.086Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4ABM7PWJPMH6E51SNF91 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T11:48:43.055" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:33.086" pid=1389 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 207.143412] gitaly[1389]: time="2026-07-06T11:48:33.086Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4ABM7PWJPMH6E51SNF91 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T11:48:43.055" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:33.086" grpc.time_ms=0.511 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.160180] gitaly[1389]: time="2026-07-06T11:48:33.099Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4ABM7PWJPMH6E51SNF91 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-06T11:48:43.060" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:33.091" grpc.time_ms=7.891 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.181941] gitaly[1389]: time="2026-07-06T11:48:33.120Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4ABM7PWJPMH6E51SNF91 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T11:48:43.089" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:33.120" grpc.time_ms=0.161 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.199216] gitaly[1389]: time="2026-07-06T11:48:33.125Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4ABM7PWJPMH6E51SNF91 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-06T11:48:43.094" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:33.125" grpc.time_ms=0.182 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.216466] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:33.128Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"997d4bc4076daaa94a1e63ec","created_at":"2026-07-06T11:48:32.998Z","trace_propagation_headers":{"sentry-trace":"3a333f0d213849c78e500f013ad17203-7a6a7a2d3b6a414a","baggage":"sentry-trace_id=3a333f0d213849c78e500f013ad17203,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4ABM7PWJPMH6E51SNF91","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:33.013Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-997d4bc4076daaa94a1e63ec: done: 0.112405 sec","job_status":"done","queue_duration_s":0.002199,"scheduling_latency_s":0.002246,"redis_calls":28,"redis_duration_s":0.011014000000000001,"redis_read_bytes":280,"redis_write_bytes":3560,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.007905,"redis_feature_flag_read_bytes":272,"redis_feature_flag_write_bytes":2395,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001745,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001364,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":16,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":16,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.076181,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.056479804,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.112405,"completed_at":"2026-07-06T11:48:33.128Z","load_balancing_strategy":"primary","db_duration_s":0.020092,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 207.311936] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:33.129Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"dd8d6788ea22cca4d7585d8d","created_at":"2026-07-06T11:48:33.025Z","trace_propagation_headers":{"sentry-trace":"3a333f0d213849c78e500f013ad17203-7a6a7a2d3b6a414a","baggage":"sentry-trace_id=3a333f0d213849c78e500f013ad17203,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4ABM7PWJPMH6E51SNF91","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:33.031Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-dd8d6788ea22cca4d7585d8d: start","job_status":"start","queue_duration_s":0.097866,"scheduling_latency_s":0.09789} gitlab # [ 207.313324] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:33.212Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"dd8d6788ea22cca4d7585d8d","created_at":"2026-07-06T11:48:33.025Z","trace_propagation_headers":{"sentry-trace":"3a333f0d213849c78e500f013ad17203-7a6a7a2d3b6a414a","baggage":"sentry-trace_id=3a333f0d213849c78e500f013ad17203,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4ABM7PWJPMH6E51SNF91","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:33.031Z","job_size_bytes":2,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-dd8d6788ea22cca4d7585d8d: done: 0.082384 sec","job_status":"done","queue_duration_s":0.097866,"scheduling_latency_s":0.09789,"redis_calls":14,"redis_duration_s":0.003618,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000711,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00167,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001237,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.056339,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.06115917,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.082384,"completed_at":"2026-07-06T11:48:33.212Z","load_balancing_strategy":"primary","db_duration_s":0.018975,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 207.584986] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:33.540Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"dda8071d62661862e05396a6","created_at":"2026-07-06T11:48:33.530Z","trace_propagation_headers":{"sentry-trace":"3a333f0d213849c78e500f013ad17203-7a6a7a2d3b6a414a","baggage":"sentry-trace_id=3a333f0d213849c78e500f013ad17203,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4ABM7PWJPMH6E51SNF91","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:33.538Z","job_size_bytes":5,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-dda8071d62661862e05396a6: start","job_status":"start","queue_duration_s":0.002338,"scheduling_latency_s":0.002387} gitlab # [ 207.693740] gitaly[1389]: time="2026-07-06T11:48:33.649Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4ABM7PWJPMH6E51SNF91 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-06T11:48:43.007" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:33.638" grpc.time_ms=11.175 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.710335] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:33.666Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"dda8071d62661862e05396a6","created_at":"2026-07-06T11:48:33.530Z","trace_propagation_headers":{"sentry-trace":"3a333f0d213849c78e500f013ad17203-7a6a7a2d3b6a414a","baggage":"sentry-trace_id=3a333f0d213849c78e500f013ad17203,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4ABM7PWJPMH6E51SNF91","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:33.538Z","job_size_bytes":5,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-dda8071d62661862e05396a6: done: 0.12594 sec","job_status":"done","queue_duration_s":0.002338,"scheduling_latency_s":0.002387,"redis_calls":12,"redis_duration_s":0.009332,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.003212,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003113,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003007,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.086538,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000361778,"duration_s":0.12594,"completed_at":"2026-07-06T11:48:33.666Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022178,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 207.790576] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:33.667Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"6f8aec96929e32e50c7cf1c7","created_at":"2026-07-06T11:48:33.591Z","trace_propagation_headers":{"sentry-trace":"3a333f0d213849c78e500f013ad17203-7a6a7a2d3b6a414a","baggage":"sentry-trace_id=3a333f0d213849c78e500f013ad17203,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4ABM7PWJPMH6E51SNF91","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:33.600Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-6f8aec96929e32e50c7cf1c7: start","job_status":"start","queue_duration_s":0.067317,"scheduling_latency_s":0.06734} gitlab # [ 207.883202] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:33.838Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"6f8aec96929e32e50c7cf1c7","created_at":"2026-07-06T11:48:33.591Z","trace_propagation_headers":{"sentry-trace":"3a333f0d213849c78e500f013ad17203-7a6a7a2d3b6a414a","baggage":"sentry-trace_id=3a333f0d213849c78e500f013ad17203,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4ABM7PWJPMH6E51SNF91","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:33.600Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-6f8aec96929e32e50c7cf1c7: done: 0.171089 sec","job_status":"done","queue_duration_s":0.067317,"scheduling_latency_s":0.06734,"redis_calls":11,"redis_duration_s":0.003351,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000652,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001995,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000704,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.138,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.138,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.114727,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000557892,"duration_s":0.171089,"completed_at":"2026-07-06T11:48:33.838Z","load_balancing_strategy":"primary","db_duration_s":0.056281,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 207.887296] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:33.841Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"ef21252c8ceeec24dffe98b1","created_at":"2026-07-06T11:48:33.604Z","trace_propagation_headers":{"sentry-trace":"3a333f0d213849c78e500f013ad17203-7a6a7a2d3b6a414a","baggage":"sentry-trace_id=3a333f0d213849c78e500f013ad17203,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4ABM7PWJPMH6E51SNF91","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:33.615Z","job_size_bytes":89,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-ef21252c8ceeec24dffe98b1: start","job_status":"start","queue_duration_s":0.225723,"scheduling_latency_s":0.225748} gitlab # [ 207.960575] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:33.902Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"ef21252c8ceeec24dffe98b1","created_at":"2026-07-06T11:48:33.604Z","trace_propagation_headers":{"sentry-trace":"3a333f0d213849c78e500f013ad17203-7a6a7a2d3b6a414a","baggage":"sentry-trace_id=3a333f0d213849c78e500f013ad17203,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4ABM7PWJPMH6E51SNF91","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:33.615Z","job_size_bytes":89,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-ef21252c8ceeec24dffe98b1: done: 0.061199 sec","job_status":"done","queue_duration_s":0.225723,"scheduling_latency_s":0.225748,"redis_calls":11,"redis_duration_s":0.004134,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000953,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002378,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000803,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034747,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000402286,"duration_s":0.061199,"completed_at":"2026-07-06T11:48:33.902Z","load_balancing_strategy":"primary","db_duration_s":0.01609,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 208.186506] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:34 +0000] "POST /api/v4/projects HTTP/1.1" 201 4795 "" "curl/8.20.0" 3432 gitlab # [ 208.190606] setup-gitlab-start[2737]: + mkdir -p /tmp/test-flake gitlab # [ 208.200691] setup-gitlab-start[2737]: + cd /tmp/test-flake gitlab # [ 208.203929] setup-gitlab-start[2737]: + git init -b master gitlab # [ 208.230223] setup-gitlab-start[3063]: Initialized empty Git repository in /tmp/test-flake/.git/ gitlab # [ 208.233463] setup-gitlab-start[2737]: + git config user.name test gitlab # [ 208.243702] setup-gitlab-start[2737]: + git config user.email test@example.com gitlab # [ 208.251873] setup-gitlab-start[2737]: + cat gitlab # [ 208.258766] setup-gitlab-start[2737]: + git add flake.nix gitlab # [ 208.267557] setup-gitlab-start[2737]: + git commit -m 'initial commit' gitlab # [ 208.282510] setup-gitlab-start[3068]: [master (root-commit) 86156a0] initial commit gitlab # [ 208.284760] setup-gitlab-start[3068]: 1 file changed, 10 insertions(+) gitlab # [ 208.288230] setup-gitlab-start[3068]: create mode 100644 flake.nix gitlab # [ 208.290893] setup-gitlab-start[2737]: + git remote add origin http://oauth2:glpat-e1OeptbxmaRucTIydj-R9G86MQp1OjEH.01.0w1gnfhwj@gitlab/root/test-flake.git gitlab # [ 208.294314] setup-gitlab-start[2737]: + git push -u origin master gitlab # [ 208.406163] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:34 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 93 gitlab # [ 208.554516] gitaly[1389]: time="2026-07-06T11:48:34.510Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4DXYC0SBBHXPMKWK3C87 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:34.495" grpc.time_ms=14.352 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 208.596172] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:34 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.54.0" 185 gitlab # [ 208.934867] gitaly[1389]: time="2026-07-06T11:48:34.891Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-06T11:48:44.006" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:34.877" grpc.time_ms=13.842 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 208.981579] gitaly[1389]: time="2026-07-06T11:48:34.937Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=331 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-06T11:49:04.023" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-06T11:48:34.924" grpc.time_ms=13.272 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.000431] gitaly[1389]: time="2026-07-06T11:48:34.956Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KWVM4E4518MXPACCAAYWNS8E duration_ms=240ns method=POST pid=1389 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 209.026107] gitaly[1389]: time="2026-07-06T11:48:34.982Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KWVM4E4518MXPACCAAYWNS8E duration_ms=23ns method=POST pid=1389 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 209.028143] gitaly[1389]: time="2026-07-06T11:48:34.982Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5319 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T11:48:34.715" grpc.time_ms=267.636 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 209.083615] gitaly[1389]: time="2026-07-06T11:48:35.039Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4789 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T11:48:35.038" grpc.time_ms=0.809 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 209.138386] gitaly[1389]: time="2026-07-06T11:48:35.094Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4788 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T11:48:35.093" grpc.time_ms=0.561 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 209.181868] gitaly[1389]: time="2026-07-06T11:48:35.138Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4790 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T11:48:35.137" grpc.time_ms=0.588 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 209.271640] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:35.227Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 86156a038a1dc206179ff7f7af1cde0bd4389896 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"403402c72e655f00de2b2709","created_at":"2026-07-06T11:48:35.220Z","trace_propagation_headers":{"sentry-trace":"ec92a773a9584026947ce643ea8d2c5a-b1f617bc6dd8403f","baggage":"sentry-trace_id=ec92a773a9584026947ce643ea8d2c5a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4E4518MXPACCAAYWNS8E","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:35.224Z","job_size_bytes":185,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-403402c72e655f00de2b2709: start","job_status":"start","queue_duration_s":0.001953,"scheduling_latency_s":0.001998} gitlab # [ 209.307833] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:35.264Z","message":"changes: \"0000000000000000000000000000000000000000 86156a038a1dc206179ff7f7af1cde0bd4389896 refs/heads/master\\n\""} gitlab # [ 209.323101] gitaly[1389]: time="2026-07-06T11:48:35.279Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-06T11:48:45.240" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:35.271" grpc.time_ms=7.735 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.352810] gitaly[1389]: time="2026-07-06T11:48:35.309Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-06T11:48:45.002" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:35.302" grpc.time_ms=5.96 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.374692] gitaly[1389]: time="2026-07-06T11:48:35.329Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1560 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-06T11:48:45.279" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T11:48:35.310" grpc.time_ms=19.23 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.396585] gitaly[1389]: time="2026-07-06T11:48:35.347Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KWVM4E4518MXPACCAAYWNS8E duration_ms=161ns method=POST pid=1389 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 209.403203] gitaly[1389]: time="2026-07-06T11:48:35.347Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4813 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T11:48:35.184" grpc.time_ms=162.411 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 209.420441] gitaly[1389]: time="2026-07-06T11:48:35.350Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T11:48:45.035" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T11:48:35.336" grpc.time_ms=13.83 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.441255] gitaly[1389]: time="2026-07-06T11:48:35.368Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E diskcache=65253764-576a-4525-b520-d061ce64e086 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:34.632" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 209.443940] gitlab-workhorse[1293]: time="2026-07-06T11:48:35Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 209.444997] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:35 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 806 gitlab # [ 209.446154] systemd[1]: Finished setup-gitlab.service. gitlab # [ 209.447274] setup-gitlab-start[3072]: To http://gitlab/root/test-flake.git gitlab # [ 209.448115] setup-gitlab-start[3072]: * [new branch] master -> master gitlab # [ 209.448918] setup-gitlab-start[3072]: branch 'master' set up to track 'origin/master'. gitlab # [ 209.450105] gitaly[1389]: time="2026-07-06T11:48:35.368Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=252 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13470 command.oublock=32 command.real_time_ms=723 command.spawn_token_fork_ms=1 command.system_time_ms=163 command.user_time_ms=89 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:34.632" grpc.time_ms=735.853 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 209.450966] systemd[1]: Starting nixbot CI service... gitlab # [ 209.453118] gitaly[1389]: time="2026-07-06T11:48:35.399Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-06T11:48:45.089" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=226 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T11:48:35.389" grpc.time_ms=9.823 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.481157] gitaly[1389]: time="2026-07-06T11:48:35.437Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3940 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T11:48:35.436" grpc.time_ms=0.477 pid=1389 span.kind=server system=grpc gitlab # [ 209.508454] gitaly[1389]: time="2026-07-06T11:48:35.464Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3942 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T11:48:35.464" grpc.time_ms=0.358 pid=1389 span.kind=server system=grpc gitlab # [ 209.519050] gitaly[1389]: time="2026-07-06T11:48:35.475Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E diskcache=107df6a3-3327-470e-b435-177c30843221 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-06T11:48:45.110" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:35.410" pid=1389 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 209.520155] gitaly[1389]: time="2026-07-06T11:48:35.475Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=61 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5007 command.oublock=8 command.real_time_ms=57 command.spawn_token_fork_ms=0 command.system_time_ms=43 command.user_time_ms=18 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-06T11:48:45.110" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:35.410" grpc.time_ms=64.649 pid=1389 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 209.536751] gitaly[1389]: time="2026-07-06T11:48:35.492Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-06T11:48:45.192" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:35.492" grpc.time_ms=0.274 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.553537] gitaly[1389]: time="2026-07-06T11:48:35.509Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=948 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-06T11:48:45.199" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T11:48:35.499" grpc.time_ms=9.902 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.622288] gitaly[1389]: time="2026-07-06T11:48:35.578Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=339 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-07-06T11:49:05.071" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:35.571" grpc.time_ms=6.275 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: waiting for unit setup-gitlab.service, in 210.58 seconds) gitlab # [ 209.668734] gitaly[1389]: time="2026-07-06T11:48:35.625Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-06T11:48:45.004" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:35.624" grpc.time_ms=0.248 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: waiting for unit nixbot.service gitlab # [ 209.684711] gitaly[1389]: time="2026-07-06T11:48:35.632Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=344 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-06T11:48:45.007" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-07-06T11:48:35.627" grpc.time_ms=5.643 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.707139] gitaly[1389]: time="2026-07-06T11:48:35.651Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T11:48:45.023" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T11:48:35.644" grpc.time_ms=7.469 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.708158] gitaly[1389]: time="2026-07-06T11:48:35.663Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T11:48:45.036" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T11:48:35.656" grpc.time_ms=6.571 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.752575] gitaly[1389]: time="2026-07-06T11:48:35.708Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=22 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=22 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=958 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-06T11:49:05.174" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:35.675" grpc.time_ms=33.536 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 210.068265] gitaly[1389]: time="2026-07-06T11:48:36.024Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-06T11:48:46.001" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:36.022" grpc.time_ms=1.704 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 210.117711] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:36.073Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 86156a038a1dc206179ff7f7af1cde0bd4389896 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"403402c72e655f00de2b2709","created_at":"2026-07-06T11:48:35.220Z","trace_propagation_headers":{"sentry-trace":"ec92a773a9584026947ce643ea8d2c5a-b1f617bc6dd8403f","baggage":"sentry-trace_id=ec92a773a9584026947ce643ea8d2c5a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4E4518MXPACCAAYWNS8E","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:35.224Z","job_size_bytes":185,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-403402c72e655f00de2b2709: done: 0.846825 sec","job_status":"done","queue_duration_s":0.001953,"scheduling_latency_s":0.001998,"gitaly_calls":13,"gitaly_duration_s":0.258317,"redis_calls":196,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.058668,"redis_read_bytes":1445,"redis_write_bytes":19020,"redis_cache_calls":14,"redis_cache_duration_s":0.003822,"redis_cache_read_bytes":16,"redis_cache_write_bytes":884,"redis_feature_flag_calls":28,"redis_feature_flag_duration_s":0.009298,"redis_feature_flag_read_bytes":1044,"redis_feature_flag_write_bytes":3487,"redis_queues_calls":8,"redis_queues_duration_s":0.001319,"redis_queues_read_bytes":8,"redis_queues_write_bytes":4936,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002915,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1259,"redis_repository_cache_calls":91,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.022359,"redis_repository_cache_read_bytes":233,"redis_repository_cache_write_bytes":6319,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.018955,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.046,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.222,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.238,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.046,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.469997,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001005435,"exclusive_lock_hold_duration_s":0.041130926,"duration_s":0.846825,"completed_at":"2026-07-06T11:48:36.073Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.077369,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 210.156560] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:36.075Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"1bddd9d97fabe374ceecb442","created_at":"2026-07-06T11:48:35.356Z","trace_propagation_headers":{"sentry-trace":"ec92a773a9584026947ce643ea8d2c5a-e157fcb36f20426b","baggage":"sentry-trace_id=ec92a773a9584026947ce643ea8d2c5a,sentry-release=Unknown"},"correlation_id":"01KWVM4E4518MXPACCAAYWNS8E","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:35.363Z","job_size_bytes":46,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-1bddd9d97fabe374ceecb442: start","job_status":"start","queue_duration_s":0.712213,"scheduling_latency_s":0.712236} gitlab # [ 210.167634] gitaly[1389]: time="2026-07-06T11:48:36.123Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-07-06T17:48:36.117" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:36.118" grpc.time_ms=5.066 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 210.184798] gitaly[1389]: time="2026-07-06T11:48:36.140Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=647 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-06T11:49:06.032" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:36.132" grpc.time_ms=7.838 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 210.214646] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:36.170Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"1bddd9d97fabe374ceecb442","created_at":"2026-07-06T11:48:35.356Z","trace_propagation_headers":{"sentry-trace":"ec92a773a9584026947ce643ea8d2c5a-e157fcb36f20426b","baggage":"sentry-trace_id=ec92a773a9584026947ce643ea8d2c5a,sentry-release=Unknown"},"correlation_id":"01KWVM4E4518MXPACCAAYWNS8E","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:35.363Z","job_size_bytes":46,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-1bddd9d97fabe374ceecb442: done: 0.09526 sec","job_status":"done","queue_duration_s":0.712213,"scheduling_latency_s":0.712236,"gitaly_calls":2,"gitaly_duration_s":0.018729,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.01275,"redis_read_bytes":68,"redis_write_bytes":4885,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001696,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000642,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2112,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001903,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.007447,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":754,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001062,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049861,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000370159,"duration_s":0.09526,"completed_at":"2026-07-06T11:48:36.170Z","load_balancing_strategy":"primary","db_duration_s":0.014265,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 210.286732] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:36.174Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"ea9ea748224336e3360aeba5","created_at":"2026-07-06T11:48:35.377Z","trace_propagation_headers":{"sentry-trace":"ec92a773a9584026947ce643ea8d2c5a-e157fcb36f20426b","baggage":"sentry-trace_id=ec92a773a9584026947ce643ea8d2c5a,sentry-release=Unknown"},"correlation_id":"01KWVM4E4518MXPACCAAYWNS8E","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:35.384Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-ea9ea748224336e3360aeba5: start","job_status":"start","queue_duration_s":0.789585,"scheduling_latency_s":0.789608} gitlab # [ 210.313876] gitaly[1389]: time="2026-07-06T11:48:36.270Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E diskcache=a211242c-d9dd-4a1c-9841-1fe58f0de4e8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-06T17:48:36.195" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:36.196" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 210.315452] gitaly[1389]: time="2026-07-06T11:48:36.270Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1284 command.oublock=0 command.real_time_ms=117 command.spawn_token_fork_ms=3 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-06T17:48:36.195" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:36.196" grpc.time_ms=73.669 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 210.398349] gitaly[1389]: time="2026-07-06T11:48:36.335Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=649 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-06T11:49:06.227" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:36.328" grpc.time_ms=6.886 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 210.399914] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:36.301Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"ea9ea748224336e3360aeba5","created_at":"2026-07-06T11:48:35.377Z","trace_propagation_headers":{"sentry-trace":"ec92a773a9584026947ce643ea8d2c5a-e157fcb36f20426b","baggage":"sentry-trace_id=ec92a773a9584026947ce643ea8d2c5a,sentry-release=Unknown"},"correlation_id":"01KWVM4E4518MXPACCAAYWNS8E","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:35.384Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-ea9ea748224336e3360aeba5: done: 0.127696 sec","job_status":"done","queue_duration_s":0.789585,"scheduling_latency_s":0.789608,"gitaly_calls":1,"gitaly_duration_s":0.076756,"redis_calls":15,"redis_duration_s":0.003758,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000451,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001482,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00066,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001165,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.015,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034033,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.105933576,"duration_s":0.127696,"completed_at":"2026-07-06T11:48:36.301Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012309,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 210.477109] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:36.303Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"961bf06b920c23688f0ab34c","created_at":"2026-07-06T11:48:35.488Z","trace_propagation_headers":{"sentry-trace":"ec92a773a9584026947ce643ea8d2c5a-e157fcb36f20426b","baggage":"sentry-trace_id=ec92a773a9584026947ce643ea8d2c5a,sentry-release=Unknown"},"correlation_id":"01KWVM4E4518MXPACCAAYWNS8E","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:35.489Z","job_size_bytes":23,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-961bf06b920c23688f0ab34c: start","job_status":"start","queue_duration_s":0.813587,"scheduling_latency_s":0.813611} gitlab # [ 210.478425] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:36.346Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"961bf06b920c23688f0ab34c","created_at":"2026-07-06T11:48:35.488Z","trace_propagation_headers":{"sentry-trace":"ec92a773a9584026947ce643ea8d2c5a-e157fcb36f20426b","baggage":"sentry-trace_id=ec92a773a9584026947ce643ea8d2c5a,sentry-release=Unknown"},"correlation_id":"01KWVM4E4518MXPACCAAYWNS8E","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:35.489Z","job_size_bytes":23,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-961bf06b920c23688f0ab34c: done: 0.043229 sec","job_status":"done","queue_duration_s":0.813587,"scheduling_latency_s":0.813611,"gitaly_calls":1,"gitaly_duration_s":0.009956,"redis_calls":16,"redis_duration_s":0.004299,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000395,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001308,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001799,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000797,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025705,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.043229,"completed_at":"2026-07-06T11:48:36.346Z","load_balancing_strategy":"primary","db_duration_s":0.002349,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 210.532382] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:36.348Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"8e30012e3684207d41ea3211","created_at":"2026-07-06T11:48:35.982Z","trace_propagation_headers":{"sentry-trace":"ec92a773a9584026947ce643ea8d2c5a-e157fcb36f20426b","baggage":"sentry-trace_id=ec92a773a9584026947ce643ea8d2c5a,sentry-release=Unknown"},"correlation_id":"01KWVM4E4518MXPACCAAYWNS8E","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:88314ebcc874810c90da4ecf40ba00e4c4ccf89e4b00628aa75db47e2ac56ab0","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:35.987Z","job_size_bytes":467,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-8e30012e3684207d41ea3211: start","job_status":"start","queue_duration_s":0.360433,"scheduling_latency_s":0.36046} gitlab # [ 210.611165] gitaly[1389]: time="2026-07-06T11:48:36.387Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:06.286" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:36.387" grpc.time_ms=0.645 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 210.612501] gitaly[1389]: time="2026-07-06T11:48:36.399Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-06T11:48:46.009" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T11:48:36.390" grpc.time_ms=8.881 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 210.675122] gitaly[1389]: time="2026-07-06T11:48:36.630Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=22 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=22 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-06T11:49:06.507" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:36.607" grpc.time_ms=22.849 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 210.683344] gitaly[1389]: time="2026-07-06T11:48:36.639Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=63 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-06T11:49:06.536" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:36.636" grpc.time_ms=2.817 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 210.753078] nixbot[3171]: {"ts": "2026-07-06T11:48:36.708Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} gitlab # [ 210.770654] gitaly[1389]: time="2026-07-06T11:48:36.726Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T11:48:46.098" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T11:48:36.719" grpc.time_ms=7.448 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 210.778344] nixbot[3171]: {"ts": "2026-07-06T11:48:36.734Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} gitlab # [ 210.792743] nixbot[3171]: {"ts": "2026-07-06T11:48:36.749Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} gitlab # [ 210.797223] nixbot[3171]: {"ts": "2026-07-06T11:48:36.753Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} gitlab # [ 210.804462] nixbot[3171]: {"ts": "2026-07-06T11:48:36.760Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} gitlab # [ 210.816062] nixbot[3171]: {"ts": "2026-07-06T11:48:36.770Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} gitlab # [ 210.817879] nixbot[3171]: {"ts": "2026-07-06T11:48:36.774Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} gitlab # [ 210.820251] nixbot[3171]: {"ts": "2026-07-06T11:48:36.776Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} gitlab # [ 210.825088] nixbot[3171]: {"ts": "2026-07-06T11:48:36.781Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} gitlab # [ 210.827076] nixbot[3171]: {"ts": "2026-07-06T11:48:36.783Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} gitlab # [ 210.831391] nixbot[3171]: {"ts": "2026-07-06T11:48:36.787Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} gitlab # [ 210.836293] nixbot[3171]: {"ts": "2026-07-06T11:48:36.792Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} gitlab # [ 210.839961] nixbot[3171]: {"ts": "2026-07-06T11:48:36.796Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} gitlab # [ 210.846337] nixbot[3171]: {"ts": "2026-07-06T11:48:36.802Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} gitlab # [ 210.850476] nixbot[3171]: {"ts": "2026-07-06T11:48:36.806Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} gitlab # [ 210.854614] nixbot[3171]: {"ts": "2026-07-06T11:48:36.808Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} gitlab # [ 210.858313] nixbot[3171]: {"ts": "2026-07-06T11:48:36.814Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} gitlab # [ 210.864521] nixbot[3171]: {"ts": "2026-07-06T11:48:36.820Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} gitlab # [ 210.872411] nixbot[3171]: {"ts": "2026-07-06T11:48:36.828Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} gitlab # [ 210.878501] nixbot[3171]: {"ts": "2026-07-06T11:48:36.834Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"} gitlab # [ 210.884348] nixbot[3171]: {"ts": "2026-07-06T11:48:36.840Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"} gitlab # [ 211.215917] nixbot[3171]: INFO: Started server process [3171] gitlab # [ 211.219851] nixbot[3171]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit) gitlab # [ 211.227828] nixbot[3171]: INFO: ::ffff:127.0.0.1:59072 - "GET /health HTTP/1.1" 200 OK gitlab # [ 211.234581] systemd[1]: Started nixbot CI service. gitlab # [ 211.242774] systemd[1]: Reached target Multi-User System. gitlab # [ 211.246875] systemd[1]: Startup finished in 1.698s (kernel) + 4.275s (initrd) + 3min 25.271s (userspace) = 3min 31.246s. gitlab # [ 211.427434] gitaly[1389]: time="2026-07-06T11:48:37.383Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=962 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-06T11:49:07.262" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:37.363" grpc.time_ms=19.775 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.449529] gitaly[1389]: time="2026-07-06T11:48:37.405Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4E4518MXPACCAAYWNS8E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-06T11:49:07.286" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:37.386" grpc.time_ms=18.644 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.517751] sidekiq-cluster[2163]: {"severity":"WARN","time":"2026-07-06T11:48:37.473Z","class":"CreatePipelineWorker","correlation_id":"01KWVM4E4518MXPACCAAYWNS8E","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"86156a038a1dc206179ff7f7af1cde0bd4389896","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"86156a038a1dc206179ff7f7af1cde0bd4389896","ref":"refs/heads/master"},"retry":0} gitlab # [ 211.529495] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:37.483Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"8e30012e3684207d41ea3211","created_at":"2026-07-06T11:48:35.982Z","trace_propagation_headers":{"sentry-trace":"ec92a773a9584026947ce643ea8d2c5a-e157fcb36f20426b","baggage":"sentry-trace_id=ec92a773a9584026947ce643ea8d2c5a,sentry-release=Unknown"},"correlation_id":"01KWVM4E4518MXPACCAAYWNS8E","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:88314ebcc874810c90da4ecf40ba00e4c4ccf89e4b00628aa75db47e2ac56ab0","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:35.987Z","job_size_bytes":467,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-8e30012e3684207d41ea3211: done: 1.135324 sec","job_status":"done","queue_duration_s":0.360433,"scheduling_latency_s":0.36046,"gitaly_calls":7,"gitaly_duration_s":0.123749,"redis_calls":53,"redis_duration_s":0.047681,"redis_read_bytes":171,"redis_write_bytes":4427,"redis_cache_calls":4,"redis_cache_duration_s":0.00425,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.006176,"redis_feature_flag_read_bytes":18,"redis_feature_flag_write_bytes":2433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003313,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.032828,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001114,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":40,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":40,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.059,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.053,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.840284,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000473244,"duration_s":1.135324,"completed_at":"2026-07-06T11:48:37.483Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.089498,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 211.606300] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:37.561Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"3d674e6e80952e0e677923f2","created_at":"2026-07-06T11:48:36.152Z","trace_propagation_headers":{"sentry-trace":"ec92a773a9584026947ce643ea8d2c5a-de43d2e958654c6b","baggage":"sentry-trace_id=ec92a773a9584026947ce643ea8d2c5a,sentry-release=Unknown"},"correlation_id":"01KWVM4E4518MXPACCAAYWNS8E","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:36.159Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-3d674e6e80952e0e677923f2: start","job_status":"start","queue_duration_s":1.402605,"scheduling_latency_s":1.402682} gitlab # [ 211.666130] gitaly[1389]: time="2026-07-06T11:48:37.622Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=965 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4GNSKTS59DDJJRPP0JSQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-06T11:49:07.005" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:37.606" grpc.time_ms=15.989 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.676699] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:37.632Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"3d674e6e80952e0e677923f2","created_at":"2026-07-06T11:48:36.152Z","trace_propagation_headers":{"sentry-trace":"ec92a773a9584026947ce643ea8d2c5a-de43d2e958654c6b","baggage":"sentry-trace_id=ec92a773a9584026947ce643ea8d2c5a,sentry-release=Unknown"},"correlation_id":"01KWVM4E4518MXPACCAAYWNS8E","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:36.159Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-3d674e6e80952e0e677923f2: done: 0.071233 sec","job_status":"done","queue_duration_s":1.402605,"scheduling_latency_s":1.402682,"redis_calls":19,"redis_duration_s":0.007018,"redis_read_bytes":21,"redis_write_bytes":4292,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001524,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.001117,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2263,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.003239,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001138,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043422,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.011678299,"duration_s":0.071233,"completed_at":"2026-07-06T11:48:37.632Z","load_balancing_strategy":"primary","db_duration_s":0.009925,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 211.691075] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:37.634Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"4ef5d9f447ca4eacdb65f875","created_at":"2026-07-06T11:48:37.613Z","trace_propagation_headers":{"sentry-trace":"ec92a773a9584026947ce643ea8d2c5a-4c4ffe315398458b","baggage":"sentry-trace_id=ec92a773a9584026947ce643ea8d2c5a,sentry-release=Unknown"},"correlation_id":"01KWVM4E4518MXPACCAAYWNS8E","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:37.620Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-4ef5d9f447ca4eacdb65f875: start","job_status":"start","queue_duration_s":0.014105,"scheduling_latency_s":0.014128} gitlab: (finished: waiting for unit nixbot.service, in 2.09 seconds) gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health gitlab # [ 211.756958] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:37.713Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"4ef5d9f447ca4eacdb65f875","created_at":"2026-07-06T11:48:37.613Z","trace_propagation_headers":{"sentry-trace":"ec92a773a9584026947ce643ea8d2c5a-4c4ffe315398458b","baggage":"sentry-trace_id=ec92a773a9584026947ce643ea8d2c5a,sentry-release=Unknown"},"correlation_id":"01KWVM4E4518MXPACCAAYWNS8E","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:37.620Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-4ef5d9f447ca4eacdb65f875: done: 0.078687 sec","job_status":"done","queue_duration_s":0.014105,"scheduling_latency_s":0.014128,"redis_calls":11,"redis_duration_s":0.003883,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000692,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002525,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000666,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.052158,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000468495,"duration_s":0.078687,"completed_at":"2026-07-06T11:48:37.712Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019298,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 211.889137] nixbot[3171]: INFO: ::ffff:127.0.0.1:59080 - "GET /health HTTP/1.1" 200 OK gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.04 seconds) subtest: project discovered and webhook registered gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 211.996523] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:37 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.20.0" 72 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.11 seconds) gitlab # [ 212.271749] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:38 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 1050 gitlab # [ 212.282598] nixbot[3171]: {"ts": "2026-07-06T11:48:38.238Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 212.324972] nixbot[3171]: {"ts": "2026-07-06T11:48:38.281Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} gitlab # [ 212.406727] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:38 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 67 gitlab # [ 212.408819] nixbot[3171]: {"ts": "2026-07-06T11:48:38.364Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 212.411645] nixbot[3171]: {"ts": "2026-07-06T11:48:38.367Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} gitlab # [ 212.535500] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:38 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 868 "" "python-httpx/0.28.1" 103 gitlab # [ 212.539189] nixbot[3171]: {"ts": "2026-07-06T11:48:38.492Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""} gitlab # [ 212.692167] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:38 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 141 gitlab # [ 212.701830] nixbot[3171]: {"ts": "2026-07-06T11:48:38.649Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 212.752975] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:38 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "python-httpx/0.28.1" 45 gitlab # [ 212.761603] nixbot[3171]: {"ts": "2026-07-06T11:48:38.711Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 212.889527] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:38 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 868 "" "python-httpx/0.28.1" 126 gitlab # [ 212.893853] nixbot[3171]: {"ts": "2026-07-06T11:48:38.846Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""} gitlab # [ 212.951608] gitaly[1389]: time="2026-07-06T11:48:38.906Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4JA4K8X8MGV8M9ZAK02P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-07-06T11:49:08.098" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T11:48:38.899" grpc.time_ms=7.233 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.963692] gitaly[1389]: time="2026-07-06T11:48:38.919Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4JA4K8X8MGV8M9ZAK02P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:08.117" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:38.918" grpc.time_ms=1.611 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 212.982427] gitaly[1389]: time="2026-07-06T11:48:38.938Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4JA4K8X8MGV8M9ZAK02P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-06T11:48:48.216" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:38.927" grpc.time_ms=10.879 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.060849] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 731 "" "python-httpx/0.28.1" 163 gitlab # [ 213.065622] nixbot[3171]: {"ts": "2026-07-06T11:48:39.017Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""} gitlab # [ 213.104968] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "curl/8.20.0" 69 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.10 seconds) (finished: subtest: project discovered and webhook registered, in 1.22 seconds) subtest: push delivers the webhook gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master gitlab # [ 213.165197] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 99 gitlab # [ 213.166462] nixbot[3171]: {"ts": "2026-07-06T11:48:39.122Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 213.169749] nixbot[3171]: {"ts": "2026-07-06T11:48:39.126Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} gitlab # [ 213.187043] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:39 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 43 gitlab # [ 213.252315] gitaly[1389]: time="2026-07-06T11:48:39.207Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4JKNDS7JWHEVFNKDZ99P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:39.199" grpc.time_ms=7.737 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 213.253884] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:39 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 51 gitlab # [ 213.313231] gitaly[1389]: time="2026-07-06T11:48:39.267Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4JKC6459G9CYKVWVFY9F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:39.260" grpc.time_ms=7.455 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 213.314512] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:39 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.54.0" 120 gitlab # Enumerating objects: 5, done. gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. gitlab # Delta compression using up to 4 threads[ 213.350224] gitaly[1389]: time="2026-07-06T11:48:39.306Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4JNBANXPMXP5AGAY0QPB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:39.290" pid=1389 remote_ip="2001:db8:1::1" request_sha=083c43590fffa527704b32b8836424e8ed055017 response_bytes=149 span.kind=server system=grpc gitlab # [ 213.352796] gitaly[1389]: time="2026-07-06T11:48:39.308Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4JNBANXPMXP5AGAY0QPB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:39.290" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 213.353909] gitaly[1389]: time="2026-07-06T11:48:39.309Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4JNBANXPMXP5AGAY0QPB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:39.290" grpc.time_ms=18.243 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 213.355079] gitlab-workhorse[1293]: time="2026-07-06T11:48:39Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=149 gitlab # [ 213.355945] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:39 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 149 "" "git/2.54.0" 98 gitlab # gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 137.00 KiB/s, done. gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) gitlab # [ 213.462703] gitaly[1389]: time="2026-07-06T11:48:39.418Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4JRVFFWA26XXA7BGWBNW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:39.403" pid=1389 remote_ip="2001:db8:1::1" request_sha=56b31007c73b9459d8cdd82c701cd5382532a9a4 response_bytes=224 span.kind=server system=grpc gitlab # [ 213.464662] gitaly[1389]: time="2026-07-06T11:48:39.418Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1202 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4JRVFFWA26XXA7BGWBNW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:39.403" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 213.467227] gitlab-workhorse[1293]: time="2026-07-06T11:48:39Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=224 gitlab # [ 213.467972] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:39 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 224 "" "git/2.54.0" 96 gitlab # [ 213.470237] gitaly[1389]: time="2026-07-06T11:48:39.418Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1202 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4JRVFFWA26XXA7BGWBNW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:39.403" grpc.time_ms=15.759 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 213.644195] gitaly[1389]: time="2026-07-06T11:48:39.600Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=5 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4JZ2W80V2V5NPHKB7BZT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:39.587" grpc.time_ms=13.088 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 213.645618] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:39 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 78 gitlab # [ 213.705226] gitaly[1389]: time="2026-07-06T11:48:39.661Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=3 command.system_time_ms=1 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4JTNNYK3NPS9D1B2VSB5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-06T11:48:49.014" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=253 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:39.646" grpc.time_ms=14.728 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.737550] gitaly[1389]: time="2026-07-06T11:48:39.691Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=330 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=4 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4JTNNYK3NPS9D1B2VSB5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-06T11:49:09.080" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-06T11:48:39.680" grpc.time_ms=11.133 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.788432] gitaly[1389]: time="2026-07-06T11:48:39.744Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KWVM4JTNNYK3NPS9D1B2VSB5 duration_ms=202ns method=POST pid=1389 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 213.814150] gitaly[1389]: time="2026-07-06T11:48:39.769Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4K1KAGZD4W88MKSRMMBE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:39.752" pid=1389 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc gitlab # [ 213.816358] gitaly[1389]: time="2026-07-06T11:48:39.769Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=3 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4K1KAGZD4W88MKSRMMBE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:39.752" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 213.817471] gitlab-workhorse[1293]: time="2026-07-06T11:48:39Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 213.818355] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:39 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 167 gitlab # [ 213.819213] gitaly[1389]: time="2026-07-06T11:48:39.770Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=3 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4K1KAGZD4W88MKSRMMBE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:39.752" grpc.time_ms=17.535 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 213.892786] gitaly[1389]: time="2026-07-06T11:48:39.848Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KWVM4JTNNYK3NPS9D1B2VSB5 duration_ms=98ns method=POST pid=1389 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 213.898385] gitaly[1389]: time="2026-07-06T11:48:39.854Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVM4JTNNYK3NPS9D1B2VSB5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5319 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T11:48:39.540" grpc.time_ms=310.05 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 213.947545] gitaly[1389]: time="2026-07-06T11:48:39.903Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVM4JTNNYK3NPS9D1B2VSB5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4789 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T11:48:39.900" grpc.time_ms=0.563 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 213.998536] gitaly[1389]: time="2026-07-06T11:48:39.954Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVM4JTNNYK3NPS9D1B2VSB5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4788 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T11:48:39.953" grpc.time_ms=0.465 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 214.015237] gitaly[1389]: time="2026-07-06T11:48:39.965Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4KADB69G6PC4VTVRRCYG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:39.958" grpc.time_ms=7.049 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 214.016350] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:39 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 81 gitlab # [ 214.038781] gitaly[1389]: time="2026-07-06T11:48:39.995Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVM4JTNNYK3NPS9D1B2VSB5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4790 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T11:48:39.994" grpc.time_ms=0.508 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 214.099474] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:40.055Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","86156a038a1dc206179ff7f7af1cde0bd4389896 1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"eedb8aa8ded06229ffd9cd17","created_at":"2026-07-06T11:48:40.050Z","trace_propagation_headers":{"sentry-trace":"63c6490dff9f4d66b5e4ff95589a226b-369d07f32f924ce1","baggage":"sentry-trace_id=63c6490dff9f4d66b5e4ff95589a226b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4JTNNYK3NPS9D1B2VSB5","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:40.051Z","job_size_bytes":185,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-eedb8aa8ded06229ffd9cd17: start","job_status":"start","queue_duration_s":0.003137,"scheduling_latency_s":0.003183} gitlab # [ 214.117386] gitaly[1389]: time="2026-07-06T11:48:40.073Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KWVM4JTNNYK3NPS9D1B2VSB5 duration_ms=48ns method=POST pid=1389 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 214.119321] gitaly[1389]: time="2026-07-06T11:48:40.075Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVM4JTNNYK3NPS9D1B2VSB5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4813 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T11:48:40.024" grpc.time_ms=51.166 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 214.128821] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:40.085Z","message":"changes: \"86156a038a1dc206179ff7f7af1cde0bd4389896 1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc refs/heads/master\\n\""} gitlab # [ 214.133781] gitaly[1389]: time="2026-07-06T11:48:40.088Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KWVM4JTNNYK3NPS9D1B2VSB5 diskcache=6ad17377-060b-428f-b1fb-974d3e6d4273 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:39.463" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 214.135721] gitlab-workhorse[1293]: time="2026-07-06T11:48:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 214.136835] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:40 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 707 gitlab # [ 214.137693] gitaly[1389]: time="2026-07-06T11:48:40.088Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=170 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13313 command.oublock=32 command.real_time_ms=614 command.spawn_token_fork_ms=1 command.system_time_ms=117 command.user_time_ms=53 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4JTNNYK3NPS9D1B2VSB5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:39.463" grpc.time_ms=625.053 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 214.157368] gitaly[1389]: time="2026-07-06T11:48:40.113Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4JTNNYK3NPS9D1B2VSB5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T11:48:50.084" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T11:48:40.104" grpc.time_ms=8.814 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # To http://gitlab/root/test-flake.git gitlab # 86156a0..1eac317 master -> master gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.12 seconds) gitlab: must succeed: git -C /tmp/test-flake rev-parse master gitlab # [ 214.231676] gitaly[1389]: time="2026-07-06T11:48:40.187Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4JTNNYK3NPS9D1B2VSB5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-06T11:48:50.159" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:40.180" grpc.time_ms=7.537 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.256927] gitaly[1389]: time="2026-07-06T11:48:40.209Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4KEFY36QW3SYZTXJNFBW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:40.192" pid=1389 remote_ip="2001:db8:1::1" request_sha=69dcef142412daaffeeda28f6446e517599d516e response_bytes=214 span.kind=server system=grpc gitlab # [ 214.273188] gitlab-workhorse[1293]: time="2026-07-06T11:48:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=214 gitlab # [ 214.276868] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:40 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 214 "" "git/2.54.0" 195 gitlab # [ 214.280363] gitaly[1389]: time="2026-07-06T11:48:40.209Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1164 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4KEFY36QW3SYZTXJNFBW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:40.192" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 214.302363] gitaly[1389]: time="2026-07-06T11:48:40.209Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1164 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4KEFY36QW3SYZTXJNFBW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:40.192" grpc.time_ms=17.528 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.10 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc' gitlab # [ 214.325273] gitaly[1389]: time="2026-07-06T11:48:40.222Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4JTNNYK3NPS9D1B2VSB5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-06T11:48:50.183" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T11:48:40.203" grpc.time_ms=17.927 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.346991] gitaly[1389]: time="2026-07-06T11:48:40.236Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=653 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4JTNNYK3NPS9D1B2VSB5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-07-06T11:49:10.125" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:40.226" grpc.time_ms=9.872 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.349164] gitaly[1389]: time="2026-07-06T11:48:40.249Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=346 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4JTNNYK3NPS9D1B2VSB5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-06T11:48:50.224" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-07-06T11:48:40.245" grpc.time_ms=4.064 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.349970] gitaly[1389]: time="2026-07-06T11:48:40.267Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4JTNNYK3NPS9D1B2VSB5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T11:48:50.240" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T11:48:40.260" grpc.time_ms=6.681 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.382650] gitaly[1389]: time="2026-07-06T11:48:40.338Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4KPJ6YZ425P3WBYAFFF0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:10.037" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:40.337" grpc.time_ms=0.831 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.455174] nixbot[3171]: INFO: ::ffff:127.0.0.1:39660 - "GET /api/repos/gitlab/root/test-flake/builds?commit=1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc', in 0.13 seconds) gitlab # [ 214.467077] gitaly[1389]: time="2026-07-06T11:48:40.423Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=648 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4KPJ6YZ425P3WBYAFFF0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-07-06T11:49:10.114" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T11:48:40.414" grpc.time_ms=8.54 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.504840] gitaly[1389]: time="2026-07-06T11:48:40.461Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4KPJ6YZ425P3WBYAFFF0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T11:48:50.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T11:48:40.454" grpc.time_ms=6.664 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.528834] gitaly[1389]: time="2026-07-06T11:48:40.485Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4JTNNYK3NPS9D1B2VSB5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-06T11:48:50.083" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:40.464" grpc.time_ms=20.856 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.568657] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:40.524Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","86156a038a1dc206179ff7f7af1cde0bd4389896 1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"eedb8aa8ded06229ffd9cd17","created_at":"2026-07-06T11:48:40.050Z","trace_propagation_headers":{"sentry-trace":"63c6490dff9f4d66b5e4ff95589a226b-369d07f32f924ce1","baggage":"sentry-trace_id=63c6490dff9f4d66b5e4ff95589a226b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4JTNNYK3NPS9D1B2VSB5","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-06T11:48:40.051Z","job_size_bytes":185,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-eedb8aa8ded06229ffd9cd17: done: 0.469738 sec","job_status":"done","queue_duration_s":0.003137,"scheduling_latency_s":0.003183,"gitaly_calls":7,"gitaly_duration_s":0.154482,"redis_calls":112,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.04451,"redis_read_bytes":350,"redis_write_bytes":13070,"redis_cache_calls":9,"redis_cache_duration_s":0.002233,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000446,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":323,"redis_queues_calls":8,"redis_queues_duration_s":0.001364,"redis_queues_read_bytes":8,"redis_queues_write_bytes":6770,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.003101,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1182,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.009055,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.028311,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":29,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.035,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":29,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.029,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.029,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.035,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.215777,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000652038,"exclusive_lock_hold_duration_s":0.078032264,"duration_s":0.469738,"completed_at":"2026-07-06T11:48:40.524Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.043745,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 214.608279] gitaly[1389]: time="2026-07-06T11:48:40.564Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4KPJ6YZ425P3WBYAFFF0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-06T11:48:50.113" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:40.564" grpc.time_ms=0.27 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.665758] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:40.621Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"632586a87501e427bce38e32","created_at":"2026-07-06T11:48:40.191Z","trace_propagation_headers":{"sentry-trace":"63c6490dff9f4d66b5e4ff95589a226b-66b1522eefd84d7d","baggage":"sentry-trace_id=63c6490dff9f4d66b5e4ff95589a226b,sentry-release=Unknown"},"correlation_id":"01KWVM4JTNNYK3NPS9D1B2VSB5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:40.193Z","job_size_bytes":46,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-632586a87501e427bce38e32: start","job_status":"start","queue_duration_s":0.428239,"scheduling_latency_s":0.428286} gitlab # [ 214.688779] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:40.644Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"632586a87501e427bce38e32","created_at":"2026-07-06T11:48:40.191Z","trace_propagation_headers":{"sentry-trace":"63c6490dff9f4d66b5e4ff95589a226b-66b1522eefd84d7d","baggage":"sentry-trace_id=63c6490dff9f4d66b5e4ff95589a226b,sentry-release=Unknown"},"correlation_id":"01KWVM4JTNNYK3NPS9D1B2VSB5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:40.193Z","job_size_bytes":46,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-632586a87501e427bce38e32: done: 0.02331 sec","job_status":"done","queue_duration_s":0.428239,"scheduling_latency_s":0.428286,"redis_calls":9,"redis_duration_s":0.003188,"redis_read_bytes":23,"redis_write_bytes":782,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001398,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000263,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001527,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":304,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013555,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02331,"completed_at":"2026-07-06T11:48:40.644Z","load_balancing_strategy":"primary","db_duration_s":0.001786,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 214.691834] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:40.648Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"c56e263b29844b8abd775945","created_at":"2026-07-06T11:48:40.198Z","trace_propagation_headers":{"sentry-trace":"63c6490dff9f4d66b5e4ff95589a226b-66b1522eefd84d7d","baggage":"sentry-trace_id=63c6490dff9f4d66b5e4ff95589a226b,sentry-release=Unknown"},"correlation_id":"01KWVM4JTNNYK3NPS9D1B2VSB5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:40.199Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-c56e263b29844b8abd775945: start","job_status":"start","queue_duration_s":0.448077,"scheduling_latency_s":0.4481} gitlab # [ 214.714179] gitaly[1389]: time="2026-07-06T11:48:40.649Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=3 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4KPJ6YZ425P3WBYAFFF0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-06T11:48:50.182" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T11:48:40.633" grpc.time_ms=15.334 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.751551] gitaly[1389]: time="2026-07-06T11:48:40.707Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVM4KPJ6YZ425P3WBYAFFF0 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4157 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T11:48:40.706" grpc.time_ms=1.253 pid=1389 span.kind=server system=grpc gitlab # [ 214.789157] gitaly[1389]: time="2026-07-06T11:48:40.745Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4JTNNYK3NPS9D1B2VSB5 diskcache=d9f95041-e02f-4f7e-9678-596a9d751a44 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-06T17:48:40.669" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:40.670" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.790582] gitaly[1389]: time="2026-07-06T11:48:40.745Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=5 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1639 command.oublock=0 command.real_time_ms=107 command.spawn_token_fork_ms=51 command.system_time_ms=8 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4JTNNYK3NPS9D1B2VSB5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-06T17:48:40.669" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:40.670" grpc.time_ms=75.368 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.808143] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:40.764Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"c56e263b29844b8abd775945","created_at":"2026-07-06T11:48:40.198Z","trace_propagation_headers":{"sentry-trace":"63c6490dff9f4d66b5e4ff95589a226b-66b1522eefd84d7d","baggage":"sentry-trace_id=63c6490dff9f4d66b5e4ff95589a226b,sentry-release=Unknown"},"correlation_id":"01KWVM4JTNNYK3NPS9D1B2VSB5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:40.199Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-c56e263b29844b8abd775945: done: 0.116081 sec","job_status":"done","queue_duration_s":0.448077,"scheduling_latency_s":0.4481,"gitaly_calls":1,"gitaly_duration_s":0.079828,"redis_calls":12,"redis_duration_s":0.003828,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001498,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000879,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001451,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025563,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.096737155,"duration_s":0.116081,"completed_at":"2026-07-06T11:48:40.764Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006731,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 214.859417] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:40.765Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"091d6e655ddbba79d47d0c22","created_at":"2026-07-06T11:48:40.323Z","trace_propagation_headers":{"sentry-trace":"63c6490dff9f4d66b5e4ff95589a226b-66b1522eefd84d7d","baggage":"sentry-trace_id=63c6490dff9f4d66b5e4ff95589a226b,sentry-release=Unknown"},"correlation_id":"01KWVM4JTNNYK3NPS9D1B2VSB5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0001e6d9e6cb7278f31d91610871c35d82506b5b6d94fe5a90b2e203a9bf40fb","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:40.325Z","job_size_bytes":467,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-091d6e655ddbba79d47d0c22: start","job_status":"start","queue_duration_s":0.440633,"scheduling_latency_s":0.440657} gitlab # [ 214.886375] gitaly[1389]: time="2026-07-06T11:48:40.795Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4JTNNYK3NPS9D1B2VSB5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:10.693" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:40.794" grpc.time_ms=0.632 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.905308] gitaly[1389]: time="2026-07-06T11:48:40.806Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVM4KPJ6YZ425P3WBYAFFF0 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4159 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T11:48:40.804" grpc.time_ms=1.384 pid=1389 span.kind=server system=grpc gitlab # [ 214.992271] gitaly[1389]: time="2026-07-06T11:48:40.948Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=150 command.spawn_token_fork_ms=145 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4JTNNYK3NPS9D1B2VSB5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-06T11:48:50.176" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T11:48:40.797" grpc.time_ms=151.165 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.997940] gitaly[1389]: time="2026-07-06T11:48:40.954Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4KPJ6YZ425P3WBYAFFF0 diskcache=ac2bfdfb-7284-46b5-a0ac-46d7bca03671 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-06T11:48:50.203" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:40.657" pid=1389 ref=refs/keep-around/86156a038a1dc206179ff7f7af1cde0bd4389896 remote_ip="2001:db8:1::1" revision=86156a038a1dc206179ff7f7af1cde0bd4389896 span.kind=server system=grpc username=root gitlab # [ 214.999620] gitaly[1389]: time="2026-07-06T11:48:40.954Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=87 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=856064 command.minflt=5560 command.oublock=8 command.real_time_ms=285 command.spawn_token_fork_ms=4 command.system_time_ms=56 command.user_time_ms=31 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4KPJ6YZ425P3WBYAFFF0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-06T11:48:50.203" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:40.657" grpc.time_ms=296.656 pid=1389 ref=refs/keep-around/86156a038a1dc206179ff7f7af1cde0bd4389896 remote_ip="2001:db8:1::1" revision=86156a038a1dc206179ff7f7af1cde0bd4389896 span.kind=server system=grpc username=root gitlab # [ 215.003191] gitaly[1389]: time="2026-07-06T11:48:40.959Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4KPJ6YZ425P3WBYAFFF0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:10.058" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:40.958" grpc.time_ms=0.443 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 215.124517] gitaly[1389]: time="2026-07-06T11:48:41.080Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4JTNNYK3NPS9D1B2VSB5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-06T11:49:11.078" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:41.079" grpc.time_ms=1.03 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 215.132251] gitaly[1389]: time="2026-07-06T11:48:41.088Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=19 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4JTNNYK3NPS9D1B2VSB5 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-06T11:49:11.085" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:41.085" grpc.time_ms=2.836 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 215.179851] gitaly[1389]: time="2026-07-06T11:48:41.136Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4JTNNYK3NPS9D1B2VSB5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T11:48:51.108" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T11:48:41.129" grpc.time_ms=6.887 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 215.226286] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:41 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/86156a038a1dc206179ff7f7af1cde0bd4389896 HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 907 gitlab # [ 215.231205] nixbot[3171]: {"ts": "2026-07-06T11:48:41.187Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/86156a038a1dc206179ff7f7af1cde0bd4389896 \"HTTP/1.1 201 Created\""} gitlab # [ 215.238987] nixbot[3171]: {"ts": "2026-07-06T11:48:41.195Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-86156a038a1d-d94f90aa", "argv0": "bwrap"} gitlab # [ 215.323187] systemd[1]: Started Nix Daemon. gitlab # [ 215.361454] nix-daemon[3476]: accepted connection from pid 3475, user nixbot gitlab # [ 215.368271] nix-daemon[3476]: accepted connection from pid 3473, user nixbot gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc' gitlab # [ 215.510315] nixbot[3171]: INFO: ::ffff:127.0.0.1:39676 - "GET /api/repos/gitlab/root/test-flake/builds?commit=1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc', in 0.06 seconds) gitlab # [ 215.679750] gitaly[1389]: time="2026-07-06T11:48:41.635Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=964 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=3 command.system_time_ms=8 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4JTNNYK3NPS9D1B2VSB5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-06T11:49:11.512" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:41.612" grpc.time_ms=22.687 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 215.732111] gitaly[1389]: time="2026-07-06T11:48:41.656Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4JTNNYK3NPS9D1B2VSB5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-06T11:49:11.539" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:41.639" grpc.time_ms=16.837 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 215.752841] sidekiq-cluster[2163]: {"severity":"WARN","time":"2026-07-06T11:48:41.671Z","class":"CreatePipelineWorker","correlation_id":"01KWVM4JTNNYK3NPS9D1B2VSB5","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"86156a038a1dc206179ff7f7af1cde0bd4389896","after":"1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc","ref":"refs/heads/master"},"retry":0} gitlab # [ 215.764076] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:41.676Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"091d6e655ddbba79d47d0c22","created_at":"2026-07-06T11:48:40.323Z","trace_propagation_headers":{"sentry-trace":"63c6490dff9f4d66b5e4ff95589a226b-66b1522eefd84d7d","baggage":"sentry-trace_id=63c6490dff9f4d66b5e4ff95589a226b,sentry-release=Unknown"},"correlation_id":"01KWVM4JTNNYK3NPS9D1B2VSB5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0001e6d9e6cb7278f31d91610871c35d82506b5b6d94fe5a90b2e203a9bf40fb","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:40.325Z","job_size_bytes":467,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-091d6e655ddbba79d47d0c22: done: 0.910721 sec","job_status":"done","queue_duration_s":0.440633,"scheduling_latency_s":0.440657,"gitaly_calls":7,"gitaly_duration_s":0.304435,"redis_calls":33,"redis_duration_s":0.007492,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.001044,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001637,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003337,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001474,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":30,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.047,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":26,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.042,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.53252,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.910721,"completed_at":"2026-07-06T11:48:41.676Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.046978,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 215.838866] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:41.680Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"fbc9abd8-0101-470c-a392-1a00102d2ece\"}"],"class":"WebHookWorker","jid":"3e2477006b6f13cac79571dc","created_at":"2026-07-06T11:48:40.444Z","trace_propagation_headers":{"sentry-trace":"63c6490dff9f4d66b5e4ff95589a226b-66b1522eefd84d7d","baggage":"sentry-trace_id=63c6490dff9f4d66b5e4ff95589a226b,sentry-release=Unknown"},"correlation_id":"01KWVM4JTNNYK3NPS9D1B2VSB5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5ae100d7fa8bf073c0d4b55431feab174a25e0a628235a44e875015f60e25e43","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:40.449Z","job_size_bytes":1749,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-3e2477006b6f13cac79571dc: start","job_status":"start","queue_duration_s":1.230752,"scheduling_latency_s":1.230832} gitlab # [ 215.840871] nixbot[3171]: INFO: ::ffff:127.0.0.1:39688 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 215.847586] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:41.803Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"fbc9abd8-0101-470c-a392-1a00102d2ece\"}"],"class":"WebHookWorker","jid":"3e2477006b6f13cac79571dc","created_at":"2026-07-06T11:48:40.444Z","trace_propagation_headers":{"sentry-trace":"63c6490dff9f4d66b5e4ff95589a226b-66b1522eefd84d7d","baggage":"sentry-trace_id=63c6490dff9f4d66b5e4ff95589a226b,sentry-release=Unknown"},"correlation_id":"01KWVM4JTNNYK3NPS9D1B2VSB5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5ae100d7fa8bf073c0d4b55431feab174a25e0a628235a44e875015f60e25e43","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:40.449Z","job_size_bytes":1749,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-3e2477006b6f13cac79571dc: done: 0.123055 sec","job_status":"done","queue_duration_s":1.230752,"scheduling_latency_s":1.230832,"redis_calls":19,"redis_duration_s":0.007574,"redis_read_bytes":31,"redis_write_bytes":5282,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002124,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000334,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3644,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003214,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001902,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.012257424000011952,"cpu_s":0.084883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000424356,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","duration_s":0.123055,"completed_at":"2026-07-06T11:48:41.803Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003892,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 215.875234] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:41.805Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"8c177f5bdb1d22fbceb37381","created_at":"2026-07-06T11:48:40.964Z","trace_propagation_headers":{"sentry-trace":"8d7b11dd21c541e69f2fe3e0f3c54b39-4e9974b9b86744d6","baggage":"sentry-trace_id=8d7b11dd21c541e69f2fe3e0f3c54b39,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4KPJ6YZ425P3WBYAFFF0","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:40.972Z","job_size_bytes":65,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-8c177f5bdb1d22fbceb37381: start","job_status":"start","queue_duration_s":0.8326,"scheduling_latency_s":0.832623} gitlab # [ 215.900944] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:41.854Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"8c177f5bdb1d22fbceb37381","created_at":"2026-07-06T11:48:40.964Z","trace_propagation_headers":{"sentry-trace":"8d7b11dd21c541e69f2fe3e0f3c54b39-4e9974b9b86744d6","baggage":"sentry-trace_id=8d7b11dd21c541e69f2fe3e0f3c54b39,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4KPJ6YZ425P3WBYAFFF0","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:40.972Z","job_size_bytes":65,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-8c177f5bdb1d22fbceb37381: done: 0.049682 sec","job_status":"done","queue_duration_s":0.8326,"scheduling_latency_s":0.832623,"redis_calls":18,"redis_duration_s":0.004350000000000001,"redis_read_bytes":271,"redis_write_bytes":2103,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002262,"redis_feature_flag_read_bytes":264,"redis_feature_flag_write_bytes":1285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00168,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000408,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029772,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00039586,"duration_s":0.049682,"completed_at":"2026-07-06T11:48:41.854Z","load_balancing_strategy":"primary","db_duration_s":0.012563,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 215.971423] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:41.856Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"941eeb1d9194053b001da51b","created_at":"2026-07-06T11:48:40.974Z","trace_propagation_headers":{"sentry-trace":"8d7b11dd21c541e69f2fe3e0f3c54b39-4e9974b9b86744d6","baggage":"sentry-trace_id=8d7b11dd21c541e69f2fe3e0f3c54b39,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4KPJ6YZ425P3WBYAFFF0","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:40.977Z","job_size_bytes":65,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-941eeb1d9194053b001da51b: start","job_status":"start","queue_duration_s":0.878413,"scheduling_latency_s":0.878434} gitlab # [ 216.049913] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:41.913Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"941eeb1d9194053b001da51b","created_at":"2026-07-06T11:48:40.974Z","trace_propagation_headers":{"sentry-trace":"8d7b11dd21c541e69f2fe3e0f3c54b39-4e9974b9b86744d6","baggage":"sentry-trace_id=8d7b11dd21c541e69f2fe3e0f3c54b39,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4KPJ6YZ425P3WBYAFFF0","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:40.977Z","job_size_bytes":65,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-941eeb1d9194053b001da51b: done: 0.057066 sec","job_status":"done","queue_duration_s":0.878413,"scheduling_latency_s":0.878434,"redis_calls":15,"redis_duration_s":0.003525,"redis_read_bytes":142,"redis_write_bytes":1431,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000617,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":431,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001487,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001421,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03632,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000421841,"duration_s":0.057066,"completed_at":"2026-07-06T11:48:41.913Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010828,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 216.069462] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:41.914Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"4b76079adeb0fe88390a3c37","created_at":"2026-07-06T11:48:41.154Z","trace_propagation_headers":{"sentry-trace":"8d7b11dd21c541e69f2fe3e0f3c54b39-4e9974b9b86744d6","baggage":"sentry-trace_id=8d7b11dd21c541e69f2fe3e0f3c54b39,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4KPJ6YZ425P3WBYAFFF0","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:41.163Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4b76079adeb0fe88390a3c37: start","job_status":"start","queue_duration_s":0.751764,"scheduling_latency_s":0.751788} gitlab # [ 216.093296] gitaly[1389]: time="2026-07-06T11:48:42.000Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4KPJ6YZ425P3WBYAFFF0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-06T11:49:11.899" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:41.999" grpc.time_ms=0.838 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 216.173641] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:42 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 64 gitlab # [ 216.175124] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:42.104Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"4b76079adeb0fe88390a3c37","created_at":"2026-07-06T11:48:41.154Z","trace_propagation_headers":{"sentry-trace":"8d7b11dd21c541e69f2fe3e0f3c54b39-4e9974b9b86744d6","baggage":"sentry-trace_id=8d7b11dd21c541e69f2fe3e0f3c54b39,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4KPJ6YZ425P3WBYAFFF0","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:41.163Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4b76079adeb0fe88390a3c37: done: 0.18986 sec","job_status":"done","queue_duration_s":0.751764,"scheduling_latency_s":0.751788,"gitaly_calls":3,"gitaly_duration_s":0.013254,"redis_calls":48,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.011496,"redis_read_bytes":208,"redis_write_bytes":6803,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001398,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":16,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001839,"redis_cache_read_bytes":32,"redis_cache_write_bytes":1470,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.001708,"redis_feature_flag_read_bytes":134,"redis_feature_flag_write_bytes":992,"redis_queues_calls":2,"redis_queues_duration_s":0.000282,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.001815,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":840,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000788,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.001218,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00105,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":34,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":34,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.043,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":22,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.007,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.014,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.112559,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.163079665,"duration_s":0.18986,"completed_at":"2026-07-06T11:48:42.104Z","load_balancing_strategy":"primary","db_duration_s":0.049126,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 216.199267] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:42.106Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","3e2477006b6f13cac79571dc"],"class":"WebHooks::LogExecutionWorker","jid":"2ccb2407c377d39bb06ffa1d","created_at":"2026-07-06T11:48:41.786Z","trace_propagation_headers":{"sentry-trace":"63c6490dff9f4d66b5e4ff95589a226b-e7037f592dbd4666","baggage":"sentry-trace_id=63c6490dff9f4d66b5e4ff95589a226b,sentry-release=Unknown"},"correlation_id":"01KWVM4JTNNYK3NPS9D1B2VSB5","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1b16a9853cb04c1b0f3447e7d4b40a9d6d169a15f3dac07e3bbbafa1681bd2d","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:41.794Z","job_size_bytes":2507,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-2ccb2407c377d39bb06ffa1d: start","job_status":"start","queue_duration_s":0.311577,"scheduling_latency_s":0.3116} gitlab # [ 216.226106] gitaly[1389]: time="2026-07-06T11:48:42.016Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4NB0RZEFR4EVK20XVB4Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:41.998" grpc.time_ms=17.707 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 216.246949] gitaly[1389]: time="2026-07-06T11:48:42.057Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4KPJ6YZ425P3WBYAFFF0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:12.054" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:42.056" grpc.time_ms=0.706 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 216.266807] gitaly[1389]: time="2026-07-06T11:48:42.093Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4KPJ6YZ425P3WBYAFFF0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:12.092" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:42.092" grpc.time_ms=0.555 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 216.334916] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:42.214Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","3e2477006b6f13cac79571dc"],"class":"WebHooks::LogExecutionWorker","jid":"2ccb2407c377d39bb06ffa1d","created_at":"2026-07-06T11:48:41.786Z","trace_propagation_headers":{"sentry-trace":"63c6490dff9f4d66b5e4ff95589a226b-e7037f592dbd4666","baggage":"sentry-trace_id=63c6490dff9f4d66b5e4ff95589a226b,sentry-release=Unknown"},"correlation_id":"01KWVM4JTNNYK3NPS9D1B2VSB5","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1b16a9853cb04c1b0f3447e7d4b40a9d6d169a15f3dac07e3bbbafa1681bd2d","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:41.794Z","job_size_bytes":2507,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-2ccb2407c377d39bb06ffa1d: done: 0.108584 sec","job_status":"done","queue_duration_s":0.311577,"scheduling_latency_s":0.3116,"redis_calls":10,"redis_duration_s":0.002487,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000439,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001534,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000514,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.025,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.025,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.090827,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000631645,"duration_s":0.108584,"completed_at":"2026-07-06T11:48:42.214Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006997,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 216.355993] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:42.216Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"866936dc651cb7214974f5e1","created_at":"2026-07-06T11:48:41.982Z","trace_propagation_headers":{"sentry-trace":"8d7b11dd21c541e69f2fe3e0f3c54b39-7c9d70b9539c4ef4","baggage":"sentry-trace_id=8d7b11dd21c541e69f2fe3e0f3c54b39,sentry-release=Unknown"},"correlation_id":"01KWVM4KPJ6YZ425P3WBYAFFF0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:41.987Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-866936dc651cb7214974f5e1: start","job_status":"start","queue_duration_s":0.228733,"scheduling_latency_s":0.228758} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc' gitlab # [ 216.446197] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:42.247Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"866936dc651cb7214974f5e1","created_at":"2026-07-06T11:48:41.982Z","trace_propagation_headers":{"sentry-trace":"8d7b11dd21c541e69f2fe3e0f3c54b39-7c9d70b9539c4ef4","baggage":"sentry-trace_id=8d7b11dd21c541e69f2fe3e0f3c54b39,sentry-release=Unknown"},"correlation_id":"01KWVM4KPJ6YZ425P3WBYAFFF0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:41.987Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-866936dc651cb7214974f5e1: done: 0.031455 sec","job_status":"done","queue_duration_s":0.228733,"scheduling_latency_s":0.228758,"redis_calls":10,"redis_duration_s":0.002584,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000525,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001613,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000446,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020993,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000414578,"duration_s":0.031455,"completed_at":"2026-07-06T11:48:42.247Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003231,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 216.451123] gitaly[1389]: time="2026-07-06T11:48:42.372Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4ND40J9JXXQSMTBBJM1C grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:42.069" pid=1389 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc gitlab # [ 216.467560] gitlab-workhorse[1293]: time="2026-07-06T11:48:42Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 216.470350] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:42 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 353 gitlab # [ 216.476334] gitaly[1389]: time="2026-07-06T11:48:42.372Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=299 command.spawn_token_fork_ms=294 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4ND40J9JXXQSMTBBJM1C grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:42.069" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc', in 0.06 seconds) gitlab # [ 216.494932] gitaly[1389]: time="2026-07-06T11:48:42.373Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=299 command.spawn_token_fork_ms=294 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4ND40J9JXXQSMTBBJM1C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:42.069" grpc.time_ms=303.661 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 216.573972] gitaly[1389]: time="2026-07-06T11:48:42.451Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4NR852C94QGV541T23GN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:42.429" pid=1389 remote_ip="2001:db8:1::1" request_sha=ed16d7c60b0541b481dece5c43caa143291a6456 response_bytes=342 span.kind=server system=grpc gitlab # [ 216.576523] nixbot[3171]: INFO: ::ffff:127.0.0.1:39700 - "GET /api/repos/gitlab/root/test-flake/builds?commit=1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc HTTP/1.1" 200 OK gitlab # [ 216.577288] gitlab-workhorse[1293]: time="2026-07-06T11:48:42Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=342 gitlab # [ 216.578477] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:42 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 342 "" "git/2.54.0" 77 gitlab # [ 216.579279] gitaly[1389]: time="2026-07-06T11:48:42.451Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1197 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4NR852C94QGV541T23GN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:42.429" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 216.580516] gitaly[1389]: time="2026-07-06T11:48:42.452Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1197 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4NR852C94QGV541T23GN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:42.429" grpc.time_ms=23.233 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 216.614120] gitaly[1389]: time="2026-07-06T11:48:42.569Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4NWF608JJ2R0CV2PRWF1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:42.560" grpc.time_ms=9.142 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 216.616716] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:42 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 61 gitlab # [ 216.675492] gitaly[1389]: time="2026-07-06T11:48:42.631Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4NYE74S30K7793VP4PJF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:42.616" pid=1389 remote_ip="2001:db8:1::1" request_sha=f1e6bb59289c4d43fc3699ed3fc4760ee01ef0e9 response_bytes=216 span.kind=server system=grpc gitlab # [ 216.677364] gitaly[1389]: time="2026-07-06T11:48:42.632Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1175 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4NYE74S30K7793VP4PJF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:42.616" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 216.678579] gitaly[1389]: time="2026-07-06T11:48:42.633Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1175 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4NYE74S30K7793VP4PJF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-06T11:48:42.616" grpc.time_ms=16.393 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 216.679712] gitlab-workhorse[1293]: time="2026-07-06T11:48:42Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=216 gitlab # [ 216.680740] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:42 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.54.0" 59 gitlab # [ 216.713983] nixbot[3171]: {"ts": "2026-07-06T11:48:42.670Z", "level": "info", "logger": "nixbot.canceller", "msg": "cancelling build", "build_id": 1, "reason": "superseded"} gitlab # [ 216.721572] nix-daemon[3481]: unexpected Nix daemon error: error: interrupted by the user gitlab # [ 216.776303] gitaly[1389]: time="2026-07-06T11:48:42.732Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4P1HN9XPVJFM9PCSV2JY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:12.131" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:42.731" grpc.time_ms=0.661 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 216.812119] gitaly[1389]: time="2026-07-06T11:48:42.768Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=648 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4P1HN9XPVJFM9PCSV2JY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-07-06T11:49:12.160" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T11:48:42.760" grpc.time_ms=7.333 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 216.817163] gitaly[1389]: time="2026-07-06T11:48:42.773Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4P3154XTYJ57R239JKH6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:12.172" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:42.772" grpc.time_ms=0.82 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 216.828648] gitaly[1389]: time="2026-07-06T11:48:42.784Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4P1HN9XPVJFM9PCSV2JY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T11:48:52.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T11:48:42.775" grpc.time_ms=8.86 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 216.894970] gitaly[1389]: time="2026-07-06T11:48:42.851Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4P1HN9XPVJFM9PCSV2JY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-06T11:48:52.080" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:42.850" grpc.time_ms=0.272 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 216.923105] gitaly[1389]: time="2026-07-06T11:48:42.879Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4P1HN9XPVJFM9PCSV2JY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-06T11:48:52.102" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T11:48:42.873" grpc.time_ms=6.066 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 216.978762] gitaly[1389]: time="2026-07-06T11:48:42.934Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVM4P1HN9XPVJFM9PCSV2JY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4157 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T11:48:42.934" grpc.time_ms=0.683 pid=1389 span.kind=server system=grpc gitlab # [ 217.001459] gitaly[1389]: time="2026-07-06T11:48:42.957Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4P3154XTYJ57R239JKH6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T11:48:52.240" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T11:48:42.950" grpc.time_ms=7.077 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.020109] gitaly[1389]: time="2026-07-06T11:48:42.975Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVM4P1HN9XPVJFM9PCSV2JY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4159 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T11:48:42.973" grpc.time_ms=2.164 pid=1389 span.kind=server system=grpc gitlab # [ 217.045179] gitaly[1389]: time="2026-07-06T11:48:43.001Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4P1HN9XPVJFM9PCSV2JY diskcache=0e8193f7-2613-483e-8960-f2ae8c24603b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-06T11:48:52.122" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:42.893" pid=1389 ref=refs/keep-around/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc remote_ip="2001:db8:1::1" revision=1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc span.kind=server system=grpc username=root gitlab # [ 217.063113] gitaly[1389]: time="2026-07-06T11:48:43.019Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=71 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5293 command.oublock=8 command.real_time_ms=99 command.spawn_token_fork_ms=1 command.system_time_ms=39 command.user_time_ms=32 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4P1HN9XPVJFM9PCSV2JY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-06T11:48:52.122" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-06T11:48:42.893" grpc.time_ms=108.378 pid=1389 ref=refs/keep-around/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc remote_ip="2001:db8:1::1" revision=1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc span.kind=server system=grpc username=root gitlab # [ 217.076602] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:43.032Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"34fafbbfe6f9e6e661bfcf26","created_at":"2026-07-06T11:48:43.025Z","trace_propagation_headers":{"sentry-trace":"4a2531b4dacd4eb19f166d0b3e1e8716-d36078244e0d42e1","baggage":"sentry-trace_id=4a2531b4dacd4eb19f166d0b3e1e8716,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4P3154XTYJ57R239JKH6","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.030Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-34fafbbfe6f9e6e661bfcf26: start","job_status":"start","queue_duration_s":0.001609,"scheduling_latency_s":0.001652} gitlab # [ 217.091157] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:43 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/86156a038a1dc206179ff7f7af1cde0bd4389896 HTTP/1.1" 201 651 "" "python-httpx/0.28.1" 325 gitlab # [ 217.115035] nixbot[3171]: {"ts": "2026-07-06T11:48:43.071Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/86156a038a1dc206179ff7f7af1cde0bd4389896 \"HTTP/1.1 201 Created\""} gitlab # [ 217.121764] gitaly[1389]: time="2026-07-06T11:48:43.077Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4P1HN9XPVJFM9PCSV2JY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:13.073" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:43.076" grpc.time_ms=0.516 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.184310] gitaly[1389]: time="2026-07-06T11:48:43.140Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVM4P3154XTYJ57R239JKH6 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T11:48:43.140" grpc.time_ms=0.504 pid=1389 span.kind=server system=grpc gitlab # [ 217.204856] gitaly[1389]: time="2026-07-06T11:48:43.161Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4PE4P5D3VC0QRZA101Q1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:13.159" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:43.160" grpc.time_ms=0.514 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.245966] gitaly[1389]: time="2026-07-06T11:48:43.202Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVM4P3154XTYJ57R239JKH6 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3899 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T11:48:43.201" grpc.time_ms=0.802 pid=1389 span.kind=server system=grpc gitlab # [ 217.270133] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:43 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 552 gitlab # [ 217.275175] nixbot[3171]: {"ts": "2026-07-06T11:48:43.231Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc \"HTTP/1.1 201 Created\""} gitlab # [ 217.287643] nixbot[3171]: {"ts": "2026-07-06T11:48:43.243Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-1eac317d56f2-84c23a72", "argv0": "bwrap"} gitlab # [ 217.291815] gitaly[1389]: time="2026-07-06T11:48:43.246Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVM4P3154XTYJ57R239JKH6 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T11:48:43.245" grpc.time_ms=0.459 pid=1389 span.kind=server system=grpc gitlab # [ 217.309461] gitaly[1389]: time="2026-07-06T11:48:43.248Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4PE4P5D3VC0QRZA101Q1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T11:48:53.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T11:48:43.239" grpc.time_ms=9.307 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.339055] gitaly[1389]: time="2026-07-06T11:48:43.294Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4P3154XTYJ57R239JKH6 diskcache=fe8b1ce7-aad4-44ab-944c-348224d17341 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-06T11:49:13.015" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-06T11:48:43.116" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.340228] gitaly[1389]: time="2026-07-06T11:48:43.295Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=159 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7170 command.oublock=0 command.real_time_ms=170 command.spawn_token_fork_ms=1 command.system_time_ms=124 command.user_time_ms=35 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4P3154XTYJ57R239JKH6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-06T11:49:13.015" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T11:48:43.116" grpc.time_ms=178.951 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.353799] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:43 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/86156a038a1dc206179ff7f7af1cde0bd4389896 HTTP/1.1" 201 665 "" "python-httpx/0.28.1" 230 gitlab # [ 217.354835] nixbot[3171]: {"ts": "2026-07-06T11:48:43.309Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/86156a038a1dc206179ff7f7af1cde0bd4389896 \"HTTP/1.1 201 Created\""} gitlab # [ 217.419045] gitaly[1389]: time="2026-07-06T11:48:43.375Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=6 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=6 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4P3154XTYJ57R239JKH6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-06T11:49:13.267" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:43.368" grpc.time_ms=7.05 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.436362] nix-daemon[3476]: accepted connection from pid 3602, user nixbot gitlab # [ 217.447792] gitaly[1389]: time="2026-07-06T11:48:43.404Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4P3154XTYJ57R239JKH6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:13.302" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:43.403" grpc.time_ms=0.705 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.459192] nix-daemon[3476]: accepted connection from pid 3603, user nixbot gitlab # [ 217.486308] gitaly[1389]: time="2026-07-06T11:48:43.442Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4P3154XTYJ57R239JKH6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:13.341" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:43.441" grpc.time_ms=0.61 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.496816] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:43.452Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"e87d219290608759d24bf63c","created_at":"2026-07-06T11:48:43.450Z","trace_propagation_headers":{"sentry-trace":"4a2531b4dacd4eb19f166d0b3e1e8716-f021018d1c904b11","baggage":"sentry-trace_id=4a2531b4dacd4eb19f166d0b3e1e8716,sentry-release=Unknown"},"correlation_id":"01KWVM4P3154XTYJ57R239JKH6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"34fafbbfe6f9e6e661bfcf26","job_size_bytes":3,"pid":2163,"job_status":"deduplicated","message":"PipelineProcessWorker JID-e87d219290608759d24bf63c: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60} gitlab # [ 217.498634] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:43.454Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"34fafbbfe6f9e6e661bfcf26","created_at":"2026-07-06T11:48:43.025Z","trace_propagation_headers":{"sentry-trace":"4a2531b4dacd4eb19f166d0b3e1e8716-d36078244e0d42e1","baggage":"sentry-trace_id=4a2531b4dacd4eb19f166d0b3e1e8716,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4P3154XTYJ57R239JKH6","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.030Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2163,"job_status":"rescheduled","message":"PipelineProcessWorker JID-34fafbbfe6f9e6e661bfcf26: rescheduled"} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc', in 0.04 seconds) (finished: subtest: push delivers the webhook, in 4.50 seconds) subtest: eval and build post commit statuses gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc/statuses gitlab # [ 217.505087] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:43.461Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"34fafbbfe6f9e6e661bfcf26","created_at":"2026-07-06T11:48:43.025Z","trace_propagation_headers":{"sentry-trace":"4a2531b4dacd4eb19f166d0b3e1e8716-d36078244e0d42e1","baggage":"sentry-trace_id=4a2531b4dacd4eb19f166d0b3e1e8716,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4P3154XTYJ57R239JKH6","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.030Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-34fafbbfe6f9e6e661bfcf26: done: 0.429173 sec","job_status":"done","queue_duration_s":0.001609,"scheduling_latency_s":0.001652,"gitaly_calls":4,"gitaly_duration_s":0.197992,"redis_calls":104,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.023187,"redis_read_bytes":267,"redis_write_bytes":24006,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001171,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":17,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001779,"redis_cache_read_bytes":33,"redis_cache_write_bytes":1526,"redis_feature_flag_calls":22,"redis_feature_flag_duration_s":0.005731,"redis_feature_flag_read_bytes":22,"redis_feature_flag_write_bytes":3871,"redis_queues_calls":22,"redis_queues_duration_s":0.0037,"redis_queues_read_bytes":27,"redis_queues_write_bytes":12385,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.006038,"redis_queues_metadata_read_bytes":110,"redis_queues_metadata_write_bytes":3546,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000239,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000693,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002665,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":850,"db_count":47,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.041,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.014,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.017,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.17263,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.396324113,"duration_s":0.429173,"completed_at":"2026-07-06T11:48:43.461Z","load_balancing_strategy":"primary","db_duration_s":0.046255,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 217.566199] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:43.462Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"bcb15b37a7d69377c3c94071","created_at":"2026-07-06T11:48:43.082Z","trace_propagation_headers":{"sentry-trace":"2085fa9f5fa041d4ba22a44ad3d08bd1-1bd399da930a4e46","baggage":"sentry-trace_id=2085fa9f5fa041d4ba22a44ad3d08bd1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4P1HN9XPVJFM9PCSV2JY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.098Z","job_size_bytes":65,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-bcb15b37a7d69377c3c94071: start","job_status":"start","queue_duration_s":0.363881,"scheduling_latency_s":0.363905} gitlab # [ 217.653179] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:43.485Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"bcb15b37a7d69377c3c94071","created_at":"2026-07-06T11:48:43.082Z","trace_propagation_headers":{"sentry-trace":"2085fa9f5fa041d4ba22a44ad3d08bd1-1bd399da930a4e46","baggage":"sentry-trace_id=2085fa9f5fa041d4ba22a44ad3d08bd1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4P1HN9XPVJFM9PCSV2JY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.098Z","job_size_bytes":65,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-bcb15b37a7d69377c3c94071: done: 0.022485 sec","job_status":"done","queue_duration_s":0.363881,"scheduling_latency_s":0.363905,"redis_calls":7,"redis_duration_s":0.0018610000000000002,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001329,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000532,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014347,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022485,"completed_at":"2026-07-06T11:48:43.485Z","load_balancing_strategy":"primary","db_duration_s":0.002969,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 217.658682] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:43.486Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"8f3dcc52767a4d23e772f916","created_at":"2026-07-06T11:48:43.098Z","trace_propagation_headers":{"sentry-trace":"4a2531b4dacd4eb19f166d0b3e1e8716-f021018d1c904b11","baggage":"sentry-trace_id=4a2531b4dacd4eb19f166d0b3e1e8716,sentry-release=Unknown"},"correlation_id":"01KWVM4P3154XTYJ57R239JKH6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.104Z","job_size_bytes":23,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-8f3dcc52767a4d23e772f916: start","job_status":"start","queue_duration_s":0.382234,"scheduling_latency_s":0.382258} gitlab # [ 217.685208] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:43.598Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"8f3dcc52767a4d23e772f916","created_at":"2026-07-06T11:48:43.098Z","trace_propagation_headers":{"sentry-trace":"4a2531b4dacd4eb19f166d0b3e1e8716-f021018d1c904b11","baggage":"sentry-trace_id=4a2531b4dacd4eb19f166d0b3e1e8716,sentry-release=Unknown"},"correlation_id":"01KWVM4P3154XTYJ57R239JKH6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.104Z","job_size_bytes":23,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-8f3dcc52767a4d23e772f916: done: 0.111257 sec","job_status":"done","queue_duration_s":0.382234,"scheduling_latency_s":0.382258,"redis_calls":15,"redis_duration_s":0.008034,"redis_read_bytes":13,"redis_write_bytes":3012,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000778,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_calls":2,"redis_queues_duration_s":0.002893,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1627,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002827,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":742,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000618,"redis_rate_limiting_write_bytes":148,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000918,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":13,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":13,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.080217,"worker_id":"sidekiq_0","rate_limiting_gates":["permanent_email_failure","temporary_email_failure"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00054979,"duration_s":0.111257,"completed_at":"2026-07-06T11:48:43.598Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017832,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 217.757694] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:43.600Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"3a7df8d24b689db968338191","created_at":"2026-07-06T11:48:43.103Z","trace_propagation_headers":{"sentry-trace":"2085fa9f5fa041d4ba22a44ad3d08bd1-1bd399da930a4e46","baggage":"sentry-trace_id=2085fa9f5fa041d4ba22a44ad3d08bd1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4P1HN9XPVJFM9PCSV2JY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.109Z","job_size_bytes":65,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-3a7df8d24b689db968338191: start","job_status":"start","queue_duration_s":0.491726,"scheduling_latency_s":0.491751} gitlab # [ 217.834050] nixbot[3171]: INFO: ::ffff:127.0.0.1:39702 - "GET /api/repos/gitlab/root/test-flake/builds?commit=1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc HTTP/1.1" 200 OK gitlab # [ 217.835068] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:43.691Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"3a7df8d24b689db968338191","created_at":"2026-07-06T11:48:43.103Z","trace_propagation_headers":{"sentry-trace":"2085fa9f5fa041d4ba22a44ad3d08bd1-1bd399da930a4e46","baggage":"sentry-trace_id=2085fa9f5fa041d4ba22a44ad3d08bd1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4P1HN9XPVJFM9PCSV2JY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.109Z","job_size_bytes":65,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-3a7df8d24b689db968338191: done: 0.090384 sec","job_status":"done","queue_duration_s":0.491726,"scheduling_latency_s":0.491751,"redis_calls":10,"redis_duration_s":0.012191,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00716,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.005031,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028756,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.090384,"completed_at":"2026-07-06T11:48:43.691Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.036576,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 217.852925] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:43.694Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"92728d2aaebfead75e0000a9","created_at":"2026-07-06T11:48:43.194Z","trace_propagation_headers":{"sentry-trace":"2085fa9f5fa041d4ba22a44ad3d08bd1-1bd399da930a4e46","baggage":"sentry-trace_id=2085fa9f5fa041d4ba22a44ad3d08bd1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4P1HN9XPVJFM9PCSV2JY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.199Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-92728d2aaebfead75e0000a9: start","job_status":"start","queue_duration_s":0.495291,"scheduling_latency_s":0.495319} gitlab # [ 217.946858] gitaly[1389]: time="2026-07-06T11:48:43.903Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4PYQ3SNA9Q39TJBB62N5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:13.901" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:43.902" grpc.time_ms=0.797 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc/statuses, in 0.37 seconds) gitlab # [ 217.971202] gitaly[1389]: time="2026-07-06T11:48:43.927Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=22 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=22 catfile.request_object_count=1 catfile.request_object_ms=0 command.spawn_token_fork_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4P1HN9XPVJFM9PCSV2JY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-06T11:49:13.801" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:43.902" grpc.time_ms=25.16 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.978101] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:43 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 325 gitlab # [ 218.002127] gitaly[1389]: time="2026-07-06T11:48:43.958Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4P1HN9XPVJFM9PCSV2JY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:13.857" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:43.957" grpc.time_ms=0.684 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.050214] gitaly[1389]: time="2026-07-06T11:48:44.006Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4P1HN9XPVJFM9PCSV2JY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:14.004" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:44.005" grpc.time_ms=1.166 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.086301] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:44.040Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"92728d2aaebfead75e0000a9","created_at":"2026-07-06T11:48:43.194Z","trace_propagation_headers":{"sentry-trace":"2085fa9f5fa041d4ba22a44ad3d08bd1-1bd399da930a4e46","baggage":"sentry-trace_id=2085fa9f5fa041d4ba22a44ad3d08bd1,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4P1HN9XPVJFM9PCSV2JY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.199Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-92728d2aaebfead75e0000a9: done: 0.346045 sec","job_status":"done","queue_duration_s":0.495291,"scheduling_latency_s":0.495319,"gitaly_calls":3,"gitaly_duration_s":0.104678,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.209334,"redis_read_bytes":99,"redis_write_bytes":6013,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.093355,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.002496,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000572,"redis_queues_read_bytes":3,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003693,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00946,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000545,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005858,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":31,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":31,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":19,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.097925,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.308846236,"duration_s":0.346045,"completed_at":"2026-07-06T11:48:44.040Z","load_balancing_strategy":"primary","db_duration_s":0.030266,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 218.098785] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:44.043Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"9ed0b6c8670aafa5744e99ad","created_at":"2026-07-06T11:48:43.584Z","trace_propagation_headers":{"sentry-trace":"4a2531b4dacd4eb19f166d0b3e1e8716-be9695ee9d564e3b","baggage":"sentry-trace_id=4a2531b4dacd4eb19f166d0b3e1e8716,sentry-release=Unknown"},"correlation_id":"01KWVM4P3154XTYJ57R239JKH6","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:1022faa0a9dba0220353ecd92d9e09777c222ba24b88dd8fedcf69b28bdc875b","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.587Z","job_size_bytes":464,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-9ed0b6c8670aafa5744e99ad: start","job_status":"start","queue_duration_s":0.455582,"scheduling_latency_s":0.455606} gitlab # [ 218.128374] sidekiq-cluster[2163]: {"severity":"WARN","time":"2026-07-06T11:48:44.084Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"9ed0b6c8670aafa5744e99ad","created_at":"2026-07-06T11:48:43.584Z","trace_propagation_headers":{"sentry-trace":"4a2531b4dacd4eb19f166d0b3e1e8716-be9695ee9d564e3b","baggage":"sentry-trace_id=4a2531b4dacd4eb19f166d0b3e1e8716,sentry-release=Unknown"},"correlation_id":"01KWVM4P3154XTYJ57R239JKH6","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:1022faa0a9dba0220353ecd92d9e09777c222ba24b88dd8fedcf69b28bdc875b","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.587Z","job_size_bytes":464,"pid":2163,"sidekiq_tid":"m1cz","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-9ed0b6c8670aafa5744e99ad: fail: 0.040822 sec","job_status":"fail","queue_duration_s":0.455582,"scheduling_latency_s":0.455606,"redis_calls":7,"redis_duration_s":0.003162,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002253,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000909,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022644,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040822,"completed_at":"2026-07-06T11:48:44.083Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/pipelines.rb:33:in `pipeline_mail'","app/mailers/emails/pipelines.rb:10:in `pipeline_failed_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.00935,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 218.723722] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:44.087Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"37ee4a876ecf22034b486d33","created_at":"2026-07-06T11:48:43.303Z","trace_propagation_headers":{"sentry-trace":"4a2531b4dacd4eb19f166d0b3e1e8716-f021018d1c904b11","baggage":"sentry-trace_id=4a2531b4dacd4eb19f166d0b3e1e8716,sentry-release=Unknown"},"correlation_id":"01KWVM4P3154XTYJ57R239JKH6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.309Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mafr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-37ee4a876ecf22034b486d33: start","job_status":"start","queue_duration_s":0.778006,"scheduling_latency_s":0.778084} gitlab # [ 218.741306] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:44.219Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"37ee4a876ecf22034b486d33","created_at":"2026-07-06T11:48:43.303Z","trace_propagation_headers":{"sentry-trace":"4a2531b4dacd4eb19f166d0b3e1e8716-f021018d1c904b11","baggage":"sentry-trace_id=4a2531b4dacd4eb19f166d0b3e1e8716,sentry-release=Unknown"},"correlation_id":"01KWVM4P3154XTYJ57R239JKH6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.309Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mafr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-37ee4a876ecf22034b486d33: done: 0.13134 sec","job_status":"done","queue_duration_s":0.778006,"scheduling_latency_s":0.778084,"redis_calls":10,"redis_duration_s":0.003396,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00086,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001904,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000632,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.084,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.081,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033789,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000434133,"duration_s":0.13134,"completed_at":"2026-07-06T11:48:44.218Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.084293,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 218.861515] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:44.220Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"1f599f9da8b6ebafe3b312cd","created_at":"2026-07-06T11:48:43.326Z","trace_propagation_headers":{"sentry-trace":"4a2531b4dacd4eb19f166d0b3e1e8716-f021018d1c904b11","baggage":"sentry-trace_id=4a2531b4dacd4eb19f166d0b3e1e8716,sentry-release=Unknown"},"correlation_id":"01KWVM4P3154XTYJ57R239JKH6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.331Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mafr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-1f599f9da8b6ebafe3b312cd: start","job_status":"start","queue_duration_s":0.888937,"scheduling_latency_s":0.88896} gitlab # [ 218.862955] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:44.249Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"1f599f9da8b6ebafe3b312cd","created_at":"2026-07-06T11:48:43.326Z","trace_propagation_headers":{"sentry-trace":"4a2531b4dacd4eb19f166d0b3e1e8716-f021018d1c904b11","baggage":"sentry-trace_id=4a2531b4dacd4eb19f166d0b3e1e8716,sentry-release=Unknown"},"correlation_id":"01KWVM4P3154XTYJ57R239JKH6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.331Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mafr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-1f599f9da8b6ebafe3b312cd: done: 0.028996 sec","job_status":"done","queue_duration_s":0.888937,"scheduling_latency_s":0.88896,"redis_calls":10,"redis_duration_s":0.002236,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000427,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001377,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000432,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000348648,"duration_s":0.028996,"completed_at":"2026-07-06T11:48:44.249Z","load_balancing_strategy":"primary","db_duration_s":0.004681,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc/statuses gitlab # [ 218.901186] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:44.251Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"14dbfe903c03b6ff92ae2494","created_at":"2026-07-06T11:48:43.342Z","trace_propagation_headers":{"sentry-trace":"4a2531b4dacd4eb19f166d0b3e1e8716-f021018d1c904b11","baggage":"sentry-trace_id=4a2531b4dacd4eb19f166d0b3e1e8716,sentry-release=Unknown"},"correlation_id":"01KWVM4P3154XTYJ57R239JKH6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.346Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mafr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-14dbfe903c03b6ff92ae2494: start","job_status":"start","queue_duration_s":0.90456,"scheduling_latency_s":0.904581} gitlab # [ 218.925178] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:44.277Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"14dbfe903c03b6ff92ae2494","created_at":"2026-07-06T11:48:43.342Z","trace_propagation_headers":{"sentry-trace":"4a2531b4dacd4eb19f166d0b3e1e8716-f021018d1c904b11","baggage":"sentry-trace_id=4a2531b4dacd4eb19f166d0b3e1e8716,sentry-release=Unknown"},"correlation_id":"01KWVM4P3154XTYJ57R239JKH6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.346Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mafr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-14dbfe903c03b6ff92ae2494: done: 0.026404 sec","job_status":"done","queue_duration_s":0.90456,"scheduling_latency_s":0.904581,"redis_calls":13,"redis_duration_s":0.002917,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00045,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001374,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000612,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000481,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017324,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000345854,"duration_s":0.026404,"completed_at":"2026-07-06T11:48:44.277Z","load_balancing_strategy":"primary","db_duration_s":0.002967,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc/statuses, in 0.10 seconds) gitlab # [ 219.002991] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:44.278Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"961a91876e4db4a07e5b2816","created_at":"2026-07-06T11:48:43.346Z","trace_propagation_headers":{"sentry-trace":"4a2531b4dacd4eb19f166d0b3e1e8716-f021018d1c904b11","baggage":"sentry-trace_id=4a2531b4dacd4eb19f166d0b3e1e8716,sentry-release=Unknown"},"correlation_id":"01KWVM4P3154XTYJ57R239JKH6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.350Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mafr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-961a91876e4db4a07e5b2816: start","job_status":"start","queue_duration_s":0.928598,"scheduling_latency_s":0.92862} gitlab # [ 219.020121] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:44.304Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"961a91876e4db4a07e5b2816","created_at":"2026-07-06T11:48:43.346Z","trace_propagation_headers":{"sentry-trace":"4a2531b4dacd4eb19f166d0b3e1e8716-f021018d1c904b11","baggage":"sentry-trace_id=4a2531b4dacd4eb19f166d0b3e1e8716,sentry-release=Unknown"},"correlation_id":"01KWVM4P3154XTYJ57R239JKH6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.350Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mafr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-961a91876e4db4a07e5b2816: done: 0.026025 sec","job_status":"done","queue_duration_s":0.928598,"scheduling_latency_s":0.92862,"redis_calls":10,"redis_duration_s":0.002692,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000431,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001615,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000646,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016219,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000431619,"duration_s":0.026025,"completed_at":"2026-07-06T11:48:44.304Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002978,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 219.145524] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:44.306Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"4d1b609c818d1825c480f24f","created_at":"2026-07-06T11:48:43.350Z","trace_propagation_headers":{"sentry-trace":"4a2531b4dacd4eb19f166d0b3e1e8716-f021018d1c904b11","baggage":"sentry-trace_id=4a2531b4dacd4eb19f166d0b3e1e8716,sentry-release=Unknown"},"correlation_id":"01KWVM4P3154XTYJ57R239JKH6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.354Z","job_size_bytes":19,"pid":2163,"sidekiq_tid":"mafr","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-4d1b609c818d1825c480f24f: start","job_status":"start","queue_duration_s":0.951918,"scheduling_latency_s":0.951975} gitlab # [ 219.146921] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:44.333Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"4d1b609c818d1825c480f24f","created_at":"2026-07-06T11:48:43.350Z","trace_propagation_headers":{"sentry-trace":"4a2531b4dacd4eb19f166d0b3e1e8716-f021018d1c904b11","baggage":"sentry-trace_id=4a2531b4dacd4eb19f166d0b3e1e8716,sentry-release=Unknown"},"correlation_id":"01KWVM4P3154XTYJ57R239JKH6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.354Z","job_size_bytes":19,"pid":2163,"sidekiq_tid":"mafr","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-4d1b609c818d1825c480f24f: done: 0.027027 sec","job_status":"done","queue_duration_s":0.951918,"scheduling_latency_s":0.951975,"redis_calls":11,"redis_duration_s":0.002545,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00056,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001546,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000439,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01706,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000389155,"duration_s":0.027027,"completed_at":"2026-07-06T11:48:44.333Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003178,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 219.182252] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:44.334Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"a033bc493f9d62dc035881f8","created_at":"2026-07-06T11:48:43.356Z","trace_propagation_headers":{"sentry-trace":"4a2531b4dacd4eb19f166d0b3e1e8716-f021018d1c904b11","baggage":"sentry-trace_id=4a2531b4dacd4eb19f166d0b3e1e8716,sentry-release=Unknown"},"correlation_id":"01KWVM4P3154XTYJ57R239JKH6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.360Z","job_size_bytes":65,"pid":2163,"sidekiq_tid":"mafr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-a033bc493f9d62dc035881f8: start","job_status":"start","queue_duration_s":0.974701,"scheduling_latency_s":0.974724} gitlab # [ 219.209179] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:44.358Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"a033bc493f9d62dc035881f8","created_at":"2026-07-06T11:48:43.356Z","trace_propagation_headers":{"sentry-trace":"4a2531b4dacd4eb19f166d0b3e1e8716-f021018d1c904b11","baggage":"sentry-trace_id=4a2531b4dacd4eb19f166d0b3e1e8716,sentry-release=Unknown"},"correlation_id":"01KWVM4P3154XTYJ57R239JKH6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.360Z","job_size_bytes":65,"pid":2163,"sidekiq_tid":"mafr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-a033bc493f9d62dc035881f8: done: 0.02308 sec","job_status":"done","queue_duration_s":0.974701,"scheduling_latency_s":0.974724,"redis_calls":10,"redis_duration_s":0.002228,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000435,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00136,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000433,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014279,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000386082,"duration_s":0.02308,"completed_at":"2026-07-06T11:48:44.357Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002781,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 219.283595] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:44.359Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"7b075c18908b0ad97a5ddd88","created_at":"2026-07-06T11:48:43.361Z","trace_propagation_headers":{"sentry-trace":"4a2531b4dacd4eb19f166d0b3e1e8716-f021018d1c904b11","baggage":"sentry-trace_id=4a2531b4dacd4eb19f166d0b3e1e8716,sentry-release=Unknown"},"correlation_id":"01KWVM4P3154XTYJ57R239JKH6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.362Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mafr","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-7b075c18908b0ad97a5ddd88: start","job_status":"start","queue_duration_s":0.996801,"scheduling_latency_s":0.996831} gitlab # [ 219.303368] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:44.383Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"7b075c18908b0ad97a5ddd88","created_at":"2026-07-06T11:48:43.361Z","trace_propagation_headers":{"sentry-trace":"4a2531b4dacd4eb19f166d0b3e1e8716-f021018d1c904b11","baggage":"sentry-trace_id=4a2531b4dacd4eb19f166d0b3e1e8716,sentry-release=Unknown"},"correlation_id":"01KWVM4P3154XTYJ57R239JKH6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.362Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mafr","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-7b075c18908b0ad97a5ddd88: done: 0.024474 sec","job_status":"done","queue_duration_s":0.996801,"scheduling_latency_s":0.996831,"redis_calls":7,"redis_duration_s":0.001766,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001335,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000431,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017499,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024474,"completed_at":"2026-07-06T11:48:44.383Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002592,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 219.424099] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:44.385Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"6784eb03b716b96e792e400d","created_at":"2026-07-06T11:48:43.405Z","trace_propagation_headers":{"sentry-trace":"4a2531b4dacd4eb19f166d0b3e1e8716-f021018d1c904b11","baggage":"sentry-trace_id=4a2531b4dacd4eb19f166d0b3e1e8716,sentry-release=Unknown"},"correlation_id":"01KWVM4P3154XTYJ57R239JKH6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.410Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mafr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-6784eb03b716b96e792e400d: start","job_status":"start","queue_duration_s":0.974884,"scheduling_latency_s":0.974908} gitlab # [ 219.424590] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:44.412Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"6784eb03b716b96e792e400d","created_at":"2026-07-06T11:48:43.405Z","trace_propagation_headers":{"sentry-trace":"4a2531b4dacd4eb19f166d0b3e1e8716-f021018d1c904b11","baggage":"sentry-trace_id=4a2531b4dacd4eb19f166d0b3e1e8716,sentry-release=Unknown"},"correlation_id":"01KWVM4P3154XTYJ57R239JKH6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.410Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mafr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-6784eb03b716b96e792e400d: done: 0.027424 sec","job_status":"done","queue_duration_s":0.974884,"scheduling_latency_s":0.974908,"redis_calls":10,"redis_duration_s":0.002193,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000415,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001338,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00044,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015447,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000346692,"duration_s":0.027424,"completed_at":"2026-07-06T11:48:44.412Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006659,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 219.462793] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:44.414Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"c37945ece96c30363caff4e8","created_at":"2026-07-06T11:48:43.410Z","trace_propagation_headers":{"sentry-trace":"4a2531b4dacd4eb19f166d0b3e1e8716-f021018d1c904b11","baggage":"sentry-trace_id=4a2531b4dacd4eb19f166d0b3e1e8716,sentry-release=Unknown"},"correlation_id":"01KWVM4P3154XTYJ57R239JKH6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.414Z","job_size_bytes":6,"pid":2163,"sidekiq_tid":"mafr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-c37945ece96c30363caff4e8: start","job_status":"start","queue_duration_s":0.999581,"scheduling_latency_s":0.999602} gitlab # [ 219.488103] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:44.436Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"c37945ece96c30363caff4e8","created_at":"2026-07-06T11:48:43.410Z","trace_propagation_headers":{"sentry-trace":"4a2531b4dacd4eb19f166d0b3e1e8716-f021018d1c904b11","baggage":"sentry-trace_id=4a2531b4dacd4eb19f166d0b3e1e8716,sentry-release=Unknown"},"correlation_id":"01KWVM4P3154XTYJ57R239JKH6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.414Z","job_size_bytes":6,"pid":2163,"sidekiq_tid":"mafr","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-c37945ece96c30363caff4e8: done: 0.022018 sec","job_status":"done","queue_duration_s":0.999581,"scheduling_latency_s":0.999602,"redis_calls":10,"redis_duration_s":0.002793,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000494,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001739,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00056,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013156,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000826641,"duration_s":0.022018,"completed_at":"2026-07-06T11:48:44.435Z","load_balancing_strategy":"primary","db_duration_s":0.001609,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 219.557646] gitaly[1389]: time="2026-07-06T11:48:44.475Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4P3154XTYJ57R239JKH6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-06T11:49:14.373" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:44.474" grpc.time_ms=1.011 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 219.577475] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:45 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 62 gitlab # [ 219.581067] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:44.438Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"7887c0169a42eaa470faed01","created_at":"2026-07-06T11:48:43.454Z","trace_propagation_headers":{"sentry-trace":"4a2531b4dacd4eb19f166d0b3e1e8716-f021018d1c904b11","baggage":"sentry-trace_id=4a2531b4dacd4eb19f166d0b3e1e8716,sentry-release=Unknown"},"correlation_id":"01KWVM4P3154XTYJ57R239JKH6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.456Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mafr","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7887c0169a42eaa470faed01: start","job_status":"start","queue_duration_s":0.98185,"scheduling_latency_s":0.981874} gitlab # [ 219.605782] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:44.510Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"7887c0169a42eaa470faed01","created_at":"2026-07-06T11:48:43.454Z","trace_propagation_headers":{"sentry-trace":"4a2531b4dacd4eb19f166d0b3e1e8716-f021018d1c904b11","baggage":"sentry-trace_id=4a2531b4dacd4eb19f166d0b3e1e8716,sentry-release=Unknown"},"correlation_id":"01KWVM4P3154XTYJ57R239JKH6","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.456Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mafr","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7887c0169a42eaa470faed01: done: 0.072496 sec","job_status":"done","queue_duration_s":0.98185,"scheduling_latency_s":0.981874,"gitaly_calls":2,"gitaly_duration_s":0.006769,"redis_calls":19,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.003322,"redis_read_bytes":42,"redis_write_bytes":1891,"redis_cache_calls":8,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000651,"redis_cache_read_bytes":16,"redis_cache_write_bytes":735,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001408,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000232,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001031,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":20,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":20,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":9,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.050041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.049164629,"duration_s":0.072496,"completed_at":"2026-07-06T11:48:44.510Z","load_balancing_strategy":"primary","db_duration_s":0.012835,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 219.709346] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:44.512Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"41f6a6e8f4c6c1c9f4b5aa8c","created_at":"2026-07-06T11:48:43.858Z","trace_propagation_headers":{"sentry-trace":"2085fa9f5fa041d4ba22a44ad3d08bd1-c41479cba2ae42de","baggage":"sentry-trace_id=2085fa9f5fa041d4ba22a44ad3d08bd1,sentry-release=Unknown"},"correlation_id":"01KWVM4P1HN9XPVJFM9PCSV2JY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.861Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mafr","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-41f6a6e8f4c6c1c9f4b5aa8c: start","job_status":"start","queue_duration_s":0.650771,"scheduling_latency_s":0.650794} gitlab # [ 219.710798] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:44.536Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"41f6a6e8f4c6c1c9f4b5aa8c","created_at":"2026-07-06T11:48:43.858Z","trace_propagation_headers":{"sentry-trace":"2085fa9f5fa041d4ba22a44ad3d08bd1-c41479cba2ae42de","baggage":"sentry-trace_id=2085fa9f5fa041d4ba22a44ad3d08bd1,sentry-release=Unknown"},"correlation_id":"01KWVM4P1HN9XPVJFM9PCSV2JY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:43.861Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mafr","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-41f6a6e8f4c6c1c9f4b5aa8c: done: 0.024442 sec","job_status":"done","queue_duration_s":0.650771,"scheduling_latency_s":0.650794,"redis_calls":7,"redis_duration_s":0.0016719999999999999,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001233,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000439,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017031,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024442,"completed_at":"2026-07-06T11:48:44.536Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002683,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 219.768965] gitaly[1389]: time="2026-07-06T11:48:44.501Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4P3154XTYJ57R239JKH6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:14.399" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:44.500" grpc.time_ms=0.691 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 219.849348] gitaly[1389]: time="2026-07-06T11:48:45.012Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4R9A1FAH7628A3125TGV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:15.010" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:45.011" grpc.time_ms=0.914 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc/statuses gitlab # [ 220.213370] gitaly[1389]: time="2026-07-06T11:48:46.169Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4SC9RHDS5F2XMSVGC73T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:16.166" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:46.167" grpc.time_ms=1.263 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 220.253111] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:46 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 119 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc/statuses, in 0.19 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc/statuses gitlab # [ 221.428512] gitaly[1389]: time="2026-07-06T11:48:47.384Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4TGZCMWDZM34PNKT3D8D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:17.381" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:47.382" grpc.time_ms=1.57 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 221.485633] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:47 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 177 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc/statuses, in 0.23 seconds) gitlab # [ 221.719694] nix-daemon[3476]: accepted connection from pid 3641, user nixbot gitlab # [ 221.805317] gitaly[1389]: time="2026-07-06T11:48:47.761Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4TX4HKPYN9E8KF5367RK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:17.759" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:47.760" grpc.time_ms=0.926 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 221.920586] gitaly[1389]: time="2026-07-06T11:48:47.876Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4TX4HKPYN9E8KF5367RK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T11:48:57.087" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T11:48:47.857" grpc.time_ms=18.842 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 221.997143] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:47.951Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"cc66ce39df6ed3d42ba7f31d","created_at":"2026-07-06T11:48:47.942Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-cc14a66f058c4021","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:47.948Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mafr","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-cc66ce39df6ed3d42ba7f31d: start","job_status":"start","queue_duration_s":0.002739,"scheduling_latency_s":0.002797} gitlab # [ 222.074621] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:48 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 377 gitlab # [ 222.083540] nixbot[3171]: {"ts": "2026-07-06T11:48:48.032Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc \"HTTP/1.1 201 Created\""} gitlab # [ 222.180135] gitaly[1389]: time="2026-07-06T11:48:48.134Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4V95X68HV05WV2A1ABBP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:18.133" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:48.133" grpc.time_ms=0.917 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 222.255980] gitaly[1389]: time="2026-07-06T11:48:48.211Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4V95X68HV05WV2A1ABBP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T11:48:58.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T11:48:48.197" grpc.time_ms=13.998 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 222.258292] gitaly[1389]: time="2026-07-06T11:48:48.212Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVM4TX4HKPYN9E8KF5367RK grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T11:48:48.211" grpc.time_ms=0.645 pid=1389 span.kind=server system=grpc gitlab # [ 222.375748] gitaly[1389]: time="2026-07-06T11:48:48.332Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVM4TX4HKPYN9E8KF5367RK grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3899 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T11:48:48.331" grpc.time_ms=0.645 pid=1389 span.kind=server system=grpc gitlab # [ 222.416846] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:48 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 334 gitlab # [ 222.427922] nixbot[3171]: {"ts": "2026-07-06T11:48:48.383Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc \"HTTP/1.1 201 Created\""} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc/statuses gitlab # [ 222.463226] gitaly[1389]: time="2026-07-06T11:48:48.418Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KWVM4TX4HKPYN9E8KF5367RK grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-06T11:48:48.416" grpc.time_ms=1.226 pid=1389 span.kind=server system=grpc gitlab # [ 222.524910] gitaly[1389]: time="2026-07-06T11:48:48.480Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4TX4HKPYN9E8KF5367RK diskcache=59bbe382-2dd5-4c3f-9cde-a491dcce2a71 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-06T11:49:18.060" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-06T11:48:48.161" pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 222.528163] gitaly[1389]: time="2026-07-06T11:48:48.481Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=209 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=823296 command.minflt=7559 command.oublock=0 command.real_time_ms=297 command.spawn_token_fork_ms=0 command.system_time_ms=150 command.user_time_ms=59 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4TX4HKPYN9E8KF5367RK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-06T11:49:18.060" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T11:48:48.161" grpc.time_ms=319.365 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 222.711791] gitaly[1389]: time="2026-07-06T11:48:48.667Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4VT3MBZ3WJ0CYSS9JNZ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:18.665" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:48.666" grpc.time_ms=1.461 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 222.730874] gitaly[1389]: time="2026-07-06T11:48:48.685Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4TX4HKPYN9E8KF5367RK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-06T11:49:18.583" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:48.684" grpc.time_ms=1.393 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 222.788805] gitaly[1389]: time="2026-07-06T11:48:48.744Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4TX4HKPYN9E8KF5367RK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:18.642" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:48.743" grpc.time_ms=1.19 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 222.888547] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:48 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 264 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc/statuses, in 0.40 seconds) gitlab # [ 222.910178] gitaly[1389]: time="2026-07-06T11:48:48.866Z" level=info msg="finished unary call with code OK" catfile.duration_ms=4 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=4 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4TX4HKPYN9E8KF5367RK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:18.759" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:48.860" grpc.time_ms=5.435 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 222.973190] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:48.928Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"9a414dd8da6abfad3b437887","created_at":"2026-07-06T11:48:48.923Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-9ec5db09f34e4ac7","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"cc66ce39df6ed3d42ba7f31d","job_size_bytes":3,"pid":2163,"job_status":"deduplicated","message":"PipelineProcessWorker JID-9a414dd8da6abfad3b437887: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60} gitlab # [ 222.975835] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:48.931Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"cc66ce39df6ed3d42ba7f31d","created_at":"2026-07-06T11:48:47.942Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-cc14a66f058c4021","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:47.948Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2163,"job_status":"rescheduled","message":"PipelineProcessWorker JID-cc66ce39df6ed3d42ba7f31d: rescheduled"} gitlab # [ 223.076135] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:49.030Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"cc66ce39df6ed3d42ba7f31d","created_at":"2026-07-06T11:48:47.942Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-cc14a66f058c4021","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:47.948Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mafr","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-cc66ce39df6ed3d42ba7f31d: done: 1.078859 sec","job_status":"done","queue_duration_s":0.002739,"scheduling_latency_s":0.002797,"gitaly_calls":4,"gitaly_duration_s":0.462819,"redis_calls":93,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.15404,"redis_read_bytes":262,"redis_write_bytes":23390,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.00272,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.010684,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00367,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":712,"redis_queues_calls":25,"redis_queues_duration_s":0.093874,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14471,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.030529,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.0013,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001529,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.007014,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":38,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":38,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.093,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":23,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.017,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.022,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.047,"db_ci_duration_s":0.047,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.312803,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.877820784,"duration_s":1.078859,"completed_at":"2026-07-06T11:48:49.030Z","load_balancing_strategy":"primary","db_duration_s":0.101571,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 223.240442] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:49.195Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"a0b25eb74b94a47af13da90a","created_at":"2026-07-06T11:48:30.603Z","trace_propagation_headers":{"sentry-trace":"b67a4b8399ea42deb141606ceaa81823-100494a126354fb2","baggage":"sentry-trace_id=b67a4b8399ea42deb141606ceaa81823,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM49PMN97G67VFG85SF0KV","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:40b2e6af08479f51c91e7033a4463da97b450de5a07f74cf2e043b05638542d6","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:48.812Z","retry_count":0,"job_size_bytes":451,"pid":2163,"sidekiq_tid":"mafr","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-a0b25eb74b94a47af13da90a: start","job_status":"start","queue_duration_s":0.383703,"scheduling_latency_s":0.383778} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc/statuses gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc/statuses, in 0.22 seconds) gitlab # [ 223.353790] sidekiq-cluster[2163]: {"severity":"WARN","time":"2026-07-06T11:48:49.308Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"a0b25eb74b94a47af13da90a","created_at":"2026-07-06T11:48:30.603Z","trace_propagation_headers":{"sentry-trace":"b67a4b8399ea42deb141606ceaa81823-100494a126354fb2","baggage":"sentry-trace_id=b67a4b8399ea42deb141606ceaa81823,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KWVM49PMN97G67VFG85SF0KV","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:40b2e6af08479f51c91e7033a4463da97b450de5a07f74cf2e043b05638542d6","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:48.812Z","retry_count":0,"job_size_bytes":451,"pid":2163,"sidekiq_tid":"mafr","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-a0b25eb74b94a47af13da90a: fail: 0.111854 sec","job_status":"fail","queue_duration_s":0.383703,"scheduling_latency_s":0.383778,"redis_calls":7,"redis_duration_s":0.006938000000000001,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005871,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001067,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03868,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.111854,"completed_at":"2026-07-06T11:48:49.307Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.007989,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 224.566088] gitaly[1389]: time="2026-07-06T11:48:50.025Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4X4WXK58XC9DZ7J4AF6G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:20.023" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:50.024" grpc.time_ms=1.189 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 224.569616] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:50 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 123 gitlab # [ 224.570630] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:49.981Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"d792d4a2d67a2d78124516bc","created_at":"2026-07-06T11:48:48.135Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-9ec5db09f34e4ac7","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:48.145Z","job_size_bytes":23,"pid":2163,"sidekiq_tid":"mapz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-d792d4a2d67a2d78124516bc: start","job_status":"start","queue_duration_s":1.83583,"scheduling_latency_s":1.835895} gitlab # [ 224.583224] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:50.316Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"d792d4a2d67a2d78124516bc","created_at":"2026-07-06T11:48:48.135Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-9ec5db09f34e4ac7","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:48.145Z","job_size_bytes":23,"pid":2163,"sidekiq_tid":"mapz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-d792d4a2d67a2d78124516bc: done: 0.335351 sec","job_status":"done","queue_duration_s":1.83583,"scheduling_latency_s":1.835895,"redis_calls":7,"redis_duration_s":0.035321,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.022853,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.012468,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.205,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.19,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057785,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.335351,"completed_at":"2026-07-06T11:48:50.316Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.203937,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 224.739162] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:50.331Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"7f463a8b7c11084b3e873dc5","created_at":"2026-07-06T11:48:48.570Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-9ec5db09f34e4ac7","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:48.583Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mapz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-7f463a8b7c11084b3e873dc5: start","job_status":"start","queue_duration_s":1.748516,"scheduling_latency_s":1.748578} gitlab # [ 224.800338] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:50.437Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"7f463a8b7c11084b3e873dc5","created_at":"2026-07-06T11:48:48.570Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-9ec5db09f34e4ac7","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:48.583Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mapz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-7f463a8b7c11084b3e873dc5: done: 0.10608 sec","job_status":"done","queue_duration_s":1.748516,"scheduling_latency_s":1.748578,"redis_calls":7,"redis_duration_s":0.013703,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.010665,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003038,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.021,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027563,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.10608,"completed_at":"2026-07-06T11:48:50.437Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020555,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc/statuses gitlab # [ 224.948275] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:50.457Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"138f7bca54e1e5c40f64924d","created_at":"2026-07-06T11:48:48.606Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-9ec5db09f34e4ac7","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:48.617Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mapz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-138f7bca54e1e5c40f64924d: start","job_status":"start","queue_duration_s":1.83981,"scheduling_latency_s":1.839873} gitlab # [ 225.002070] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:50.597Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"138f7bca54e1e5c40f64924d","created_at":"2026-07-06T11:48:48.606Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-9ec5db09f34e4ac7","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:48.617Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mapz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-138f7bca54e1e5c40f64924d: done: 0.140586 sec","job_status":"done","queue_duration_s":1.83981,"scheduling_latency_s":1.839873,"redis_calls":7,"redis_duration_s":0.026112,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.013432,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.01268,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.052,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.052,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033676,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.140586,"completed_at":"2026-07-06T11:48:50.597Z","load_balancing_strategy":"primary","db_duration_s":0.052292,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 225.139337] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:50.613Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"cd9f10667ba701761ccfd118","created_at":"2026-07-06T11:48:48.637Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-9ec5db09f34e4ac7","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:48.641Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mapz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-cd9f10667ba701761ccfd118: start","job_status":"start","queue_duration_s":1.971646,"scheduling_latency_s":1.971703} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc/statuses, in 0.32 seconds) gitlab # [ 225.248238] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:51.201Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"cd9f10667ba701761ccfd118","created_at":"2026-07-06T11:48:48.637Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-9ec5db09f34e4ac7","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:48.641Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mapz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-cd9f10667ba701761ccfd118: done: 0.588771 sec","job_status":"done","queue_duration_s":1.971646,"scheduling_latency_s":1.971703,"redis_calls":10,"redis_duration_s":0.069272,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.031504,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.034209,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003559,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037758,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.588771,"completed_at":"2026-07-06T11:48:51.201Z","load_balancing_strategy":"primary","db_duration_s":0.031483,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 225.335987] gitaly[1389]: time="2026-07-06T11:48:51.226Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4YAJHV5ZJAYDKE7EBWE6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:21.223" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:51.224" grpc.time_ms=1.573 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 225.379331] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:51 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 129 gitlab # [ 225.387802] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:51.212Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"d820e9648e05fa77e55c67cc","created_at":"2026-07-06T11:48:48.643Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-9ec5db09f34e4ac7","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:48.646Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mapz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-d820e9648e05fa77e55c67cc: start","job_status":"start","queue_duration_s":2.565814,"scheduling_latency_s":2.565867} gitlab # [ 225.566317] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:51.344Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"d820e9648e05fa77e55c67cc","created_at":"2026-07-06T11:48:48.643Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-9ec5db09f34e4ac7","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:48.646Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mapz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-d820e9648e05fa77e55c67cc: done: 0.131707 sec","job_status":"done","queue_duration_s":2.565814,"scheduling_latency_s":2.565867,"redis_calls":7,"redis_duration_s":0.010351,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00808,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002271,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.042,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.027,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027068,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.131707,"completed_at":"2026-07-06T11:48:51.343Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.041778,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 225.713222] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:51.377Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"4ba7fe63e2a0107005741129","created_at":"2026-07-06T11:48:48.647Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-9ec5db09f34e4ac7","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:48.652Z","job_size_bytes":19,"pid":2163,"sidekiq_tid":"mapz","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-4ba7fe63e2a0107005741129: start","job_status":"start","queue_duration_s":2.724799,"scheduling_latency_s":2.724854} gitlab # [ 225.713692] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:51.541Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"4ba7fe63e2a0107005741129","created_at":"2026-07-06T11:48:48.647Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-9ec5db09f34e4ac7","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:48.652Z","job_size_bytes":19,"pid":2163,"sidekiq_tid":"mapz","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-4ba7fe63e2a0107005741129: done: 0.164374 sec","job_status":"done","queue_duration_s":2.724799,"scheduling_latency_s":2.724854,"redis_calls":8,"redis_duration_s":0.043706999999999996,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.027933,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.015774,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.039,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035988,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.164374,"completed_at":"2026-07-06T11:48:51.541Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.037708,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 225.852845] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:51.550Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"0925f1abbd21896cda8edc5e","created_at":"2026-07-06T11:48:48.654Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-9ec5db09f34e4ac7","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:48.657Z","job_size_bytes":66,"pid":2163,"sidekiq_tid":"mapz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-0925f1abbd21896cda8edc5e: start","job_status":"start","queue_duration_s":2.893633,"scheduling_latency_s":2.893682} gitlab # [ 225.855761] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:51.685Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"0925f1abbd21896cda8edc5e","created_at":"2026-07-06T11:48:48.654Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-9ec5db09f34e4ac7","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:48.657Z","job_size_bytes":66,"pid":2163,"sidekiq_tid":"mapz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-0925f1abbd21896cda8edc5e: done: 0.134415 sec","job_status":"done","queue_duration_s":2.893633,"scheduling_latency_s":2.893682,"redis_calls":7,"redis_duration_s":0.031252,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.019686,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.011566,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.045,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.045,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026829,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.134415,"completed_at":"2026-07-06T11:48:51.685Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.043398,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 225.989047] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:51.708Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"27613dda55a8e935ada04462","created_at":"2026-07-06T11:48:48.660Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-9ec5db09f34e4ac7","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:48.669Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mapz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-27613dda55a8e935ada04462: start","job_status":"start","queue_duration_s":3.039034,"scheduling_latency_s":3.0391} gitlab # [ 226.001614] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:51.827Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"27613dda55a8e935ada04462","created_at":"2026-07-06T11:48:48.660Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-9ec5db09f34e4ac7","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:48.669Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mapz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-27613dda55a8e935ada04462: done: 0.119147 sec","job_status":"done","queue_duration_s":3.039034,"scheduling_latency_s":3.0391,"redis_calls":7,"redis_duration_s":0.025941,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.019271,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00667,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032994,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.119147,"completed_at":"2026-07-06T11:48:51.827Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027889,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 226.186607] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:51.833Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"136549203f8c54516a1abf37","created_at":"2026-07-06T11:48:48.758Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-9ec5db09f34e4ac7","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:48.761Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mapz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-136549203f8c54516a1abf37: start","job_status":"start","queue_duration_s":3.071427,"scheduling_latency_s":3.071476} gitlab # [ 226.189571] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:51.915Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"136549203f8c54516a1abf37","created_at":"2026-07-06T11:48:48.758Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-9ec5db09f34e4ac7","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:48.761Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mapz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-136549203f8c54516a1abf37: done: 0.082573 sec","job_status":"done","queue_duration_s":3.071427,"scheduling_latency_s":3.071476,"redis_calls":7,"redis_duration_s":0.016793,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.010315,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.006478,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025271,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.082573,"completed_at":"2026-07-06T11:48:51.915Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012976,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc/statuses gitlab # [ 226.285765] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:51.922Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"7f3a70a72aae678f36b3b3bb","created_at":"2026-07-06T11:48:48.762Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-9ec5db09f34e4ac7","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:48.766Z","job_size_bytes":6,"pid":2163,"sidekiq_tid":"mapz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-7f3a70a72aae678f36b3b3bb: start","job_status":"start","queue_duration_s":3.155994,"scheduling_latency_s":3.156056} gitlab # [ 226.429151] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:52.066Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"7f3a70a72aae678f36b3b3bb","created_at":"2026-07-06T11:48:48.762Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-9ec5db09f34e4ac7","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:48.766Z","job_size_bytes":6,"pid":2163,"sidekiq_tid":"mapz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-7f3a70a72aae678f36b3b3bb: done: 0.14492 sec","job_status":"done","queue_duration_s":3.155994,"scheduling_latency_s":3.156056,"redis_calls":7,"redis_duration_s":0.027279,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.024022,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003257,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020959,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.14492,"completed_at":"2026-07-06T11:48:52.066Z","load_balancing_strategy":"primary","db_duration_s":0.005529,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 226.453203] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:52.086Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"e9efbf9b674019776f32adb7","created_at":"2026-07-06T11:48:48.767Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-9ec5db09f34e4ac7","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:48.779Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mapz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-e9efbf9b674019776f32adb7: start","job_status":"start","queue_duration_s":3.30731,"scheduling_latency_s":3.30737} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc/statuses, in 0.27 seconds) gitlab # [ 226.486888] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:52.261Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"e9efbf9b674019776f32adb7","created_at":"2026-07-06T11:48:48.767Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-9ec5db09f34e4ac7","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:48.779Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mapz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-e9efbf9b674019776f32adb7: done: 0.175317 sec","job_status":"done","queue_duration_s":3.30731,"scheduling_latency_s":3.30737,"redis_calls":10,"redis_duration_s":0.022884,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002807,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.01767,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002407,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.044,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.037,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036473,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00341468,"duration_s":0.175317,"completed_at":"2026-07-06T11:48:52.261Z","load_balancing_strategy":"primary","db_duration_s":0.085416,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 226.652485] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:52.269Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"96af499533af76fb0b0f8220","created_at":"2026-07-06T11:48:48.932Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-9ec5db09f34e4ac7","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:48.935Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mapz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-96af499533af76fb0b0f8220: start","job_status":"start","queue_duration_s":3.334533,"scheduling_latency_s":3.334582} gitlab # [ 226.661383] gitaly[1389]: time="2026-07-06T11:48:52.563Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KWVM4TX4HKPYN9E8KF5367RK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-06T11:49:22.459" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:52.560" grpc.time_ms=2.775 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 226.689077] gitaly[1389]: time="2026-07-06T11:48:52.616Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4ZNMGHHZQA3YY9A8GT1R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:22.612" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:52.614" grpc.time_ms=1.45 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 226.733420] gitaly[1389]: time="2026-07-06T11:48:52.661Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4TX4HKPYN9E8KF5367RK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:22.556" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:52.657" grpc.time_ms=1.825 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 226.775639] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:52 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 132 gitlab # [ 226.829685] gitaly[1389]: time="2026-07-06T11:48:52.785Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM4TX4HKPYN9E8KF5367RK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:22.683" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:52.784" grpc.time_ms=1.678 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 226.881456] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:52.833Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"96af499533af76fb0b0f8220","created_at":"2026-07-06T11:48:48.932Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-9ec5db09f34e4ac7","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:48.935Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mapz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-96af499533af76fb0b0f8220: done: 0.563931 sec","job_status":"done","queue_duration_s":3.334533,"scheduling_latency_s":3.334582,"gitaly_calls":3,"gitaly_duration_s":0.03706,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.064378,"redis_read_bytes":98,"redis_write_bytes":6013,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.011135,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.011011,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.003302,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.01589,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000412,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.003115,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.008378,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.196,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":20,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.02,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.027,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.123,"db_ci_duration_s":0.073,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.213195,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.470903704,"duration_s":0.563931,"completed_at":"2026-07-06T11:48:52.833Z","load_balancing_strategy":"primary","db_duration_s":0.21816,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 227.044818] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:52.841Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"7a06ec20d25124625a5b6e74","created_at":"2026-07-06T11:48:52.467Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-59288cb098144aaa","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:52.500Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mapz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-7a06ec20d25124625a5b6e74: start","job_status":"start","queue_duration_s":0.341333,"scheduling_latency_s":0.341385} gitlab # [ 227.051176] sidekiq-cluster[2163]: {"severity":"INFO","time":"2026-07-06T11:48:52.960Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"7a06ec20d25124625a5b6e74","created_at":"2026-07-06T11:48:52.467Z","trace_propagation_headers":{"sentry-trace":"47830fa5f03b470e9a6b4d7b58a77a6d-59288cb098144aaa","baggage":"sentry-trace_id=47830fa5f03b470e9a6b4d7b58a77a6d,sentry-release=Unknown"},"correlation_id":"01KWVM4TX4HKPYN9E8KF5367RK","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-07-06T11:48:52.500Z","job_size_bytes":3,"pid":2163,"sidekiq_tid":"mapz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-7a06ec20d25124625a5b6e74: done: 0.118817 sec","job_status":"done","queue_duration_s":0.341333,"scheduling_latency_s":0.341385,"redis_calls":7,"redis_duration_s":0.016888,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.012283,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004605,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039276,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.118817,"completed_at":"2026-07-06T11:48:52.960Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024911,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 227.473493] nix-daemon[3476]: accepted connection from pid 3721, user nixbot gitlab # [ 227.640149] gitaly[1389]: time="2026-07-06T11:48:53.596Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM50KT3QA7QSEFH589TR5K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:23.594" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:53.595" grpc.time_ms=0.895 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc/statuses gitlab # [ 227.798119] gitaly[1389]: time="2026-07-06T11:48:53.753Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KWVM50KT3QA7QSEFH589TR5K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-06T11:49:03.101" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-06T11:48:53.732" grpc.time_ms=19.901 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 227.913196] gitaly[1389]: time="2026-07-06T11:48:53.867Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KWVM50V6K0E3XN8K5Z46R5AA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-06T11:49:23.865" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-06T11:48:53.866" grpc.time_ms=0.874 pid=1389 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 227.957095] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:53 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 402 gitlab # [ 227.964094] nixbot[3171]: {"ts": "2026-07-06T11:48:53.919Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc \"HTTP/1.1 201 Created\""} gitlab # [ 228.015661] gitlab-workhorse[1293]: gitlab 2001:db8:1::1 - - [2026/07/06:11:48:53 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc/statuses HTTP/1.1" 200 1328 "" "curl/8.20.0" 236 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc/statuses, in 0.31 seconds) (finished: subtest: eval and build post commit statuses, in 10.42 seconds) (finished: run the VM test script, in 263.42 seconds) test script finished in 263.48s cleanup kill QemuMachine (pid 13) gitlab # qemu-system-x86_64: terminating on signal 15 from pid 6 (/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13/bin/python3.13) (finished: cleanup, in 2.52 seconds) additionally exposed symbols: gitlab, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh [] [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-07-06T11:48:38.475Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] [] [] [] [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': [{'level': 'warning', 'message': "unable to download 'https://cache.nixos.org/…': Could not resolve hostname (6) Could not resolve host: cache.nixos.org", 'count': 1}], 'error': None, 'created_at': '2026-07-06T11:48:42.662093Z', 'started_at': '2026-07-06T11:48:43.237536Z', 'finished_at': None}] [{'id': 2, 'sha': '1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-06T11:48:43.183Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 2, 'sha': '1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-06T11:48:43.183Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 2, 'sha': '1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-06T11:48:43.183Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 2, 'sha': '1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-07-06T11:48:43.183Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 2, 'sha': '1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-06T11:48:43.183Z', 'started_at': None, 'finished_at': '2026-07-06T11:48:47.921Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-06T11:48:48.319Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 2, 'sha': '1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-06T11:48:43.183Z', 'started_at': None, 'finished_at': '2026-07-06T11:48:47.921Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-06T11:48:48.319Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 2, 'sha': '1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-06T11:48:43.183Z', 'started_at': None, 'finished_at': '2026-07-06T11:48:47.921Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-06T11:48:48.319Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 2, 'sha': '1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-06T11:48:43.183Z', 'started_at': None, 'finished_at': '2026-07-06T11:48:47.921Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-07-06T11:48:48.319Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 2, 'sha': '1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-06T11:48:43.183Z', 'started_at': None, 'finished_at': '2026-07-06T11:48:47.921Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': '1eac317d56f2d6a3a4f5017663c1d87ea7cce3bc', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-07-06T11:48:48.319Z', 'started_at': None, 'finished_at': '2026-07-06T11:48:53.823Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]