tribuchet: building on jamie Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.01 seconds) Test will time out and terminate in 3600 seconds run the VM test script start all VMs gitlab: starting vm gitlab # Disk image does not exist, creating the virtualisation disk image... gitlab: QEMU running (pid 46) gitlab # Formatting '/build/vm-state-gitlab/tmp.KLs3dBmzTx', fmt=raw size=1073741824 gitlab # mke2fs 1.47.4 (6-Mar-2025) gitlab # Discarding device blocks: 0/262144 done gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes gitlab # Filesystem UUID: 1b1fb2e7-0d67-4a09-812a-d7daad8b8b80 (finished: start all VMs, in 85.44 seconds) gitlab # Superblock backups stored on blocks: gitlab: waiting for unit setup-gitlab.service gitlab # 32768, 98304, 163840, 229376 gitlab: waiting for the VM to finish booting gitlab # gitlab # Allocating group tables: 0/8 done gitlab # Writing inode tables: 0/8 done gitlab # Creating journal (8192 blocks): done gitlab # Writing superblocks and filesystem accounting information: 0/8 done gitlab # gitlab # Virtualisation disk image created. gitlab # Creating Nix store image... gitlab # Created Nix store image. gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)... gitlab # gitlab # gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)... gitlab # gitlab # gitlab # Booting from ROM... gitlab # Probing EDD (edd=off to disable)... ok gitlab # [ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 6 17:27:09 UTC 2026 gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/rq9jh1pxl6pghpfq01jiw9c8hc19zfls-nixos-system-gitlab-test/init regInfo=/nix/store/fkyv04h5qm80yzm103sjwscgw6xicpfi-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.000000] BIOS-provided physical RAM map: gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved gitlab # [ 0.000000] NX (Execute Disable) protection: active gitlab # [ 0.000000] APIC: Static calls initialized gitlab # [ 0.000000] SMBIOS 2.8 present. gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 gitlab # [ 0.000000] DMI: Memory slots populated: 1/1 gitlab # [ 0.000000] Hypervisor detected: KVM gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 gitlab # [ 0.000001] kvm-clock: using sched offset of 575862064 cycles gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns gitlab # [ 0.000005] tsc: Detected 2400.010 MHz processor gitlab # [ 0.000812] last_pfn = 0x1c0000 max_arch_pfn = 0x10000000000 gitlab # [ 0.000849] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs gitlab # [ 0.000852] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT gitlab # [ 0.000905] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.002761] found SMP MP-table at [mem 0x000f5470-0x000f547f] gitlab # [ 0.002772] Using GB pages for direct mapping gitlab # [ 0.002859] RAMDISK: [mem 0xbe31c000-0xbffcffff] gitlab # [ 0.002868] ACPI: Early table checksum verification disabled gitlab # [ 0.002870] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) gitlab # [ 0.002874] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002878] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002886] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002888] ACPI: FACS 0x00000000BFFE0000 000040 gitlab # [ 0.002890] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002891] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002893] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002894] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2] gitlab # [ 0.002895] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e] gitlab # [ 0.002896] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] gitlab # [ 0.002896] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482] gitlab # [ 0.002897] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba] gitlab # [ 0.002897] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2] gitlab # [ 0.003388] No NUMA configuration found gitlab # [ 0.003389] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff] gitlab # [ 0.003392] NODE_DATA(0) allocated [mem 0x1bfffa780-0x1bffffcff] gitlab # [ 0.006510] Zone ranges: gitlab # [ 0.006511] DMA [mem 0x0000000000001000-0x0000000000ffffff] gitlab # [ 0.006512] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] gitlab # [ 0.006514] Normal [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.006515] Device empty gitlab # [ 0.006516] Movable zone start for each node gitlab # [ 0.006517] Early memory node ranges gitlab # [ 0.006517] node 0: [mem 0x0000000000001000-0x000000000009efff] gitlab # [ 0.006518] node 0: [mem 0x0000000000100000-0x00000000bffd7fff] gitlab # [ 0.006519] node 0: [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.006520] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff] gitlab # [ 0.006542] On node 0, zone DMA: 1 pages in unavailable ranges gitlab # [ 0.006856] On node 0, zone DMA: 97 pages in unavailable ranges gitlab # [ 0.131434] On node 0, zone Normal: 40 pages in unavailable ranges gitlab # [ 0.132462] ACPI: PM-Timer IO Port: 0x608 gitlab # [ 0.132480] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) gitlab # [ 0.132518] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 gitlab # [ 0.132521] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) gitlab # [ 0.132523] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) gitlab # [ 0.132524] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) gitlab # [ 0.132525] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) gitlab # [ 0.132526] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) gitlab # [ 0.132528] ACPI: Using ACPI (MADT) for SMP configuration information gitlab # [ 0.132530] ACPI: HPET id: 0x8086a201 base: 0xfed00000 gitlab # [ 0.132536] TSC deadline timer available gitlab # [ 0.132541] CPU topo: Max. logical packages: 1 gitlab # [ 0.132542] CPU topo: Max. logical dies: 1 gitlab # [ 0.132543] CPU topo: Max. dies per package: 1 gitlab # [ 0.132546] CPU topo: Max. threads per core: 1 gitlab # [ 0.132547] CPU topo: Num. cores per package: 4 gitlab # [ 0.132547] CPU topo: Num. threads per package: 4 gitlab # [ 0.132548] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs gitlab # [ 0.132583] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() gitlab # [ 0.132603] kvm-guest: KVM setup pv remote TLB flush gitlab # [ 0.132607] kvm-guest: setup PV sched yield gitlab # [ 0.132628] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] gitlab # [ 0.132630] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] gitlab # [ 0.132631] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff] gitlab # [ 0.132632] [mem 0xc0000000-0xfeffbfff] available for PCI devices gitlab # [ 0.132634] Booting paravirtualized kernel on KVM gitlab # [ 0.132638] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns gitlab # [ 0.137101] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 gitlab # [ 0.139704] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288 gitlab # [ 0.139756] kvm-guest: PV spinlocks enabled gitlab # [ 0.139757] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) gitlab # [ 0.139760] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/rq9jh1pxl6pghpfq01jiw9c8hc19zfls-nixos-system-gitlab-test/init regInfo=/nix/store/fkyv04h5qm80yzm103sjwscgw6xicpfi-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.139861] Unknown kernel command line parameters "regInfo=/nix/store/fkyv04h5qm80yzm103sjwscgw6xicpfi-closure-info/registration", will be passed to user space. gitlab # [ 0.139874] random: crng init done gitlab # [ 0.139875] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes gitlab # [ 0.149956] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) gitlab # [ 0.154821] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) gitlab # [ 0.154904] software IO TLB: area num 4. gitlab # [ 0.234951] Fallback order for Node 0: 0 gitlab # [ 0.234960] Built 1 zonelists, mobility grouping on. Total pages: 1572726 gitlab # [ 0.234962] Policy zone: Normal gitlab # [ 0.237709] mem auto-init: stack:all(zero), heap alloc:on, heap free:off gitlab # [ 0.248273] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 gitlab # [ 0.262937] allocated 12582912 bytes of page_ext gitlab # [ 0.273383] ftrace: allocating 48717 entries in 192 pages gitlab # [ 0.273385] ftrace: allocated 192 pages with 2 groups gitlab # [ 0.274297] Dynamic Preempt: lazy gitlab # [ 0.274493] rcu: Preemptible hierarchical RCU implementation. gitlab # [ 0.274494] rcu: RCU event tracing is enabled. gitlab # [ 0.274495] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. gitlab # [ 0.274497] Trampoline variant of Tasks RCU enabled. gitlab # [ 0.274497] Rude variant of Tasks RCU enabled. gitlab # [ 0.274498] Tracing variant of Tasks RCU enabled. gitlab # [ 0.274498] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. gitlab # [ 0.274499] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 gitlab # [ 0.274526] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.274528] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.274529] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.278976] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16 gitlab # [ 0.279308] rcu: srcu_init: Setting srcu_struct sizes based on contention. gitlab # [ 0.279317] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns gitlab # [ 0.279432] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) gitlab # [ 0.286852] Console: colour VGA+ 80x25 gitlab # [ 0.286856] printk: legacy console [tty0] enabled gitlab # [ 0.331751] printk: legacy console [ttyS0] enabled gitlab # [ 0.537338] ACPI: Core revision 20250807 gitlab # [ 0.538951] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns gitlab # [ 0.541782] APIC: Switch to symmetric I/O mode setup gitlab # [ 0.543536] x2apic enabled gitlab # [ 0.544796] APIC: Switched APIC routing to: physical x2apic gitlab # [ 0.546500] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask() gitlab # [ 0.548747] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself() gitlab # [ 0.551547] kvm-guest: setup PV IPIs gitlab # [ 0.553939] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 gitlab # [ 0.555774] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns gitlab # [ 0.558912] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400010) gitlab # [ 0.560028] x86/cpu: User Mode Instruction Prevention (UMIP) activated gitlab # [ 0.562082] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 gitlab # [ 0.563910] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 gitlab # [ 0.564915] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto gitlab # [ 0.565910] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl gitlab # [ 0.566910] Transient Scheduler Attacks: Mitigation: Clear CPU buffers gitlab # [ 0.568910] Spectre V2 : Mitigation: Enhanced / Automatic IBRS gitlab # [ 0.570910] Speculative Return Stack Overflow: Mitigation: Safe RET gitlab # [ 0.571910] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization gitlab # [ 0.572919] Spectre V2 : Enabling IBPB for BPF gitlab # [ 0.573912] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier gitlab # [ 0.575911] active return thunk: srso_alias_return_thunk gitlab # [ 0.576939] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' gitlab # [ 0.578910] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' gitlab # [ 0.579910] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' gitlab # [ 0.580910] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' gitlab # [ 0.582910] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' gitlab # [ 0.584909] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' gitlab # [ 0.586910] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' gitlab # [ 0.588909] x86/fpu: Supporting XSAVE feature 0x800: 'Control-flow User registers' gitlab # [ 0.589909] x86/fpu: Supporting XSAVE feature 0x1000: 'Control-flow Kernel registers (KVM only)' gitlab # [ 0.590910] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 gitlab # [ 0.592910] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 gitlab # [ 0.594910] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 gitlab # [ 0.595909] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 gitlab # [ 0.596910] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 gitlab # [ 0.597910] x86/fpu: xstate_offset[11]: 2440, xstate_sizes[11]: 16 gitlab # [ 0.598910] x86/fpu: xstate_offset[12]: 2456, xstate_sizes[12]: 24 gitlab # [ 0.599909] x86/fpu: Enabled xstate features 0x1ae7, context size is 2480 bytes, using 'compacted' format. gitlab # [ 0.630845] Freeing SMP alternatives memory: 44K gitlab # [ 0.631912] pid_max: default: 32768 minimum: 301 gitlab # [ 0.633954] LSM: initializing lsm=capability,landlock,yama,bpf,ima gitlab # [ 0.635917] landlock: Up and running. gitlab # [ 0.636909] Yama: becoming mindful. gitlab # [ 0.638124] LSM support for eBPF active gitlab # [ 0.639937] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.641066] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.644008] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) gitlab # [ 0.645574] Performance Events: Fam17h+ core perfctr, AMD PMU driver. gitlab # [ 0.646920] ... version: 2 gitlab # [ 0.647911] ... bit width: 48 gitlab # [ 0.648912] ... generic counters: 6 gitlab # [ 0.649911] ... generic bitmap: 000000000000003f gitlab # [ 0.650911] ... fixed-purpose counters: 0 gitlab # [ 0.651911] ... fixed-purpose bitmap: 0000000000000000 gitlab # [ 0.652912] ... value mask: 0000ffffffffffff gitlab # [ 0.653912] ... max period: 00007fffffffffff gitlab # [ 0.654911] ... global_ctrl mask: 000000000000003f gitlab # [ 0.656043] signal: max sigframe size: 3376 gitlab # [ 0.657007] rcu: Hierarchical SRCU implementation. gitlab # [ 0.657916] rcu: Max phase no-delay instances is 400. gitlab # [ 0.659112] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level gitlab # [ 0.664700] smp: Bringing up secondary CPUs ... gitlab # [ 0.666372] smpboot: x86: Booting SMP configuration: gitlab # [ 0.666914] .... node #0, CPUs: #1 #2 #3 gitlab # [ 0.670959] smp: Brought up 1 node, 4 CPUs gitlab # [ 0.672912] smpboot: Total of 4 processors activated (19200.08 BogoMIPS) gitlab # [ 0.674449] Memory: 6011288K/6290904K available (17210K kernel code, 2726K rwdata, 13576K rodata, 3640K init, 2992K bss, 260100K reserved, 0K cma-reserved) gitlab # [ 0.676910] devtmpfs: initialized gitlab # [ 0.678193] x86/mm: Memory block size: 128MB gitlab # [ 0.681958] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) gitlab # [ 0.683064] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). gitlab # [ 0.684025] pinctrl core: initialized pinctrl subsystem gitlab # [ 0.685286] PM: RTC time: 19:29:20, date: 2026-09-10 gitlab # [ 0.689122] NET: Registered PF_NETLINK/PF_ROUTE protocol family gitlab # [ 0.691322] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations gitlab # [ 0.691982] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations gitlab # [ 0.694059] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations gitlab # [ 0.694926] audit: initializing netlink subsys (disabled) gitlab # [ 0.696130] audit: type=2000 audit(1789068560.197:1): state=initialized audit_enabled=0 res=1 gitlab # [ 0.696176] thermal_sys: Registered thermal governor 'fair_share' gitlab # [ 0.697912] thermal_sys: Registered thermal governor 'bang_bang' gitlab # [ 0.698912] thermal_sys: Registered thermal governor 'step_wise' gitlab # [ 0.699912] thermal_sys: Registered thermal governor 'user_space' gitlab # [ 0.700912] thermal_sys: Registered thermal governor 'power_allocator' gitlab # [ 0.701958] cpuidle: using governor menu gitlab # [ 0.705349] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 gitlab # [ 0.707329] PCI: Using configuration type 1 for base access gitlab # [ 0.707912] PCI: Using configuration type 1 for extended access gitlab # [ 0.709063] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. gitlab # [ 0.711152] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages gitlab # [ 0.712912] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page gitlab # [ 0.713912] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages gitlab # [ 0.714911] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page gitlab # [ 0.718309] ACPI: Added _OSI(Module Device) gitlab # [ 0.719914] ACPI: Added _OSI(Processor Device) gitlab # [ 0.720912] ACPI: Added _OSI(Processor Aggregator Device) gitlab # [ 0.724992] ACPI: 1 ACPI AML tables successfully acquired and loaded gitlab # [ 0.733145] ACPI: Interpreter enabled gitlab # [ 0.733929] ACPI: PM: (supports S0 S3 S4 S5) gitlab # [ 0.734913] ACPI: Using IOAPIC for interrupt routing gitlab # [ 0.736299] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug gitlab # [ 0.742924] PCI: Using E820 reservations for host bridge windows gitlab # [ 0.747608] ACPI: Enabled 2 GPEs in block 00 to 0F gitlab # [ 0.763507] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) gitlab # [ 0.767927] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] gitlab # [ 0.774126] acpiphp: Slot [3] registered gitlab # [ 0.777000] acpiphp: Slot [4] registered gitlab # [ 0.778998] acpiphp: Slot [5] registered gitlab # [ 0.782022] acpiphp: Slot [6] registered gitlab # [ 0.784997] acpiphp: Slot [7] registered gitlab # [ 0.786961] acpiphp: Slot [8] registered gitlab # [ 0.787952] acpiphp: Slot [9] registered gitlab # [ 0.788970] acpiphp: Slot [10] registered gitlab # [ 0.790954] acpiphp: Slot [11] registered gitlab # [ 0.791956] acpiphp: Slot [12] registered gitlab # [ 0.792967] acpiphp: Slot [13] registered gitlab # [ 0.794945] acpiphp: Slot [14] registered gitlab # [ 0.795944] acpiphp: Slot [15] registered gitlab # [ 0.796944] acpiphp: Slot [16] registered gitlab # [ 0.797964] acpiphp: Slot [17] registered gitlab # [ 0.799971] acpiphp: Slot [18] registered gitlab # [ 0.800945] acpiphp: Slot [19] registered gitlab # [ 0.802060] acpiphp: Slot [20] registered gitlab # [ 0.803961] acpiphp: Slot [21] registered gitlab # [ 0.804948] acpiphp: Slot [22] registered gitlab # [ 0.805945] acpiphp: Slot [23] registered gitlab # [ 0.807946] acpiphp: Slot [24] registered gitlab # [ 0.808962] acpiphp: Slot [25] registered gitlab # [ 0.809946] acpiphp: Slot [26] registered gitlab # [ 0.811967] acpiphp: Slot [27] registered gitlab # [ 0.812959] acpiphp: Slot [28] registered gitlab # [ 0.813946] acpiphp: Slot [29] registered gitlab # [ 0.815946] acpiphp: Slot [30] registered gitlab # [ 0.816945] acpiphp: Slot [31] registered gitlab # [ 0.817966] PCI host bridge to bus 0000:00 gitlab # [ 0.819919] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] gitlab # [ 0.821913] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] gitlab # [ 0.823913] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] gitlab # [ 0.825913] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] gitlab # [ 0.827913] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 0.829913] pci_bus 0000:00: root bus resource [bus 00-ff] gitlab # [ 0.832365] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint gitlab # [ 0.835906] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint gitlab # [ 0.838846] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint gitlab # [ 0.843965] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f] gitlab # [ 0.844979] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk gitlab # [ 0.846913] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk gitlab # [ 0.848912] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk gitlab # [ 0.850912] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk gitlab # [ 0.853296] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint gitlab # [ 0.857758] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f] gitlab # [ 0.860275] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint gitlab # [ 0.863572] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI gitlab # [ 0.865927] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB gitlab # [ 0.869718] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint gitlab # [ 0.879999] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] gitlab # [ 0.881941] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] gitlab # [ 0.883965] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref] gitlab # [ 0.886303] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] gitlab # [ 0.900918] pci 0000:00:02.0: pci_fixup_video+0x0/0x100 took 14648 usecs gitlab # [ 0.904021] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.909971] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f] gitlab # [ 0.911926] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] gitlab # [ 0.912966] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref] gitlab # [ 0.915926] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] gitlab # [ 0.919385] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.925950] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f] gitlab # [ 0.926927] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] gitlab # [ 0.928965] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref] gitlab # [ 0.934141] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.948958] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf] gitlab # [ 0.952956] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] gitlab # [ 0.958054] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref] gitlab # [ 0.969183] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.983005] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df] gitlab # [ 0.986946] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] gitlab # [ 0.991025] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref] gitlab # [ 0.999126] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 1.005930] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f] gitlab # [ 1.006931] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] gitlab # [ 1.008968] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref] gitlab # [ 1.013090] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 1.018927] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff] gitlab # [ 1.020927] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] gitlab # [ 1.023980] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref] gitlab # [ 1.028219] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 1.032942] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff] gitlab # [ 1.034927] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] gitlab # [ 1.036967] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref] gitlab # [ 1.038927] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref] gitlab # [ 1.043004] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint gitlab # [ 1.048954] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] gitlab # [ 1.049966] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref] gitlab # [ 1.055002] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint gitlab # [ 1.059944] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f] gitlab # [ 1.061927] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] gitlab # [ 1.062966] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref] gitlab # [ 1.068297] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 1.073929] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f] gitlab # [ 1.074927] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff] gitlab # [ 1.076966] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref] gitlab # [ 1.086271] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 gitlab # [ 1.089375] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 gitlab # [ 1.093322] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 gitlab # [ 1.096107] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 gitlab # [ 1.098022] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 gitlab # [ 1.102918] iommu: Default domain type: Translated gitlab # [ 1.103918] iommu: DMA domain TLB invalidation policy: lazy mode gitlab # [ 1.110104] ACPI: bus type USB registered gitlab # [ 1.113172] usbcore: registered new interface driver usbfs gitlab # [ 1.117011] usbcore: registered new interface driver hub gitlab # [ 1.120972] usbcore: registered new device driver usb gitlab # [ 1.127742] NetLabel: Initializing gitlab # [ 1.130918] NetLabel: domain hash size = 128 gitlab # [ 1.133919] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO gitlab # [ 1.138150] NetLabel: unlabeled traffic allowed by default gitlab # [ 1.142951] PCI: Using ACPI for IRQ routing gitlab # [ 1.148005] pci 0000:00:02.0: vgaarb: setting as boot VGA device gitlab # [ 1.148907] pci 0000:00:02.0: vgaarb: bridge control possible gitlab # [ 1.148907] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none gitlab # [ 1.160923] vgaarb: loaded gitlab # [ 1.163366] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 gitlab # [ 1.165920] hpet0: 3 comparators, 64-bit 100.000000 MHz counter gitlab # [ 1.175140] clocksource: Switched to clocksource kvm-clock gitlab # [ 1.177467] VFS: Disk quotas dquot_6.6.0 gitlab # [ 1.180694] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) gitlab # [ 1.186481] pnp: PnP ACPI init gitlab # [ 1.190809] pnp: PnP ACPI: found 6 devices gitlab # [ 1.206913] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns gitlab # [ 1.212318] clocksource: Switched to clocksource acpi_pm gitlab # [ 1.215956] NET: Registered PF_INET protocol family gitlab # [ 1.221784] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) gitlab # [ 1.253018] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) gitlab # [ 1.259656] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) gitlab # [ 1.262061] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) gitlab # [ 1.267165] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) gitlab # [ 1.269491] TCP: Hash tables configured (established 65536 bind 65536) gitlab # [ 1.272142] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) gitlab # [ 1.274824] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 1.277148] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 1.279550] NET: Registered PF_UNIX/PF_LOCAL protocol family gitlab # [ 1.281351] NET: Registered PF_XDP protocol family gitlab # [ 1.282928] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] gitlab # [ 1.284801] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] gitlab # [ 1.286644] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] gitlab # [ 1.288690] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] gitlab # [ 1.290719] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 1.293039] pci 0000:00:01.0: PIIX3: Enabling Passive Release gitlab # [ 1.294850] pci 0000:00:00.0: Limiting direct PCI/PCI transfers gitlab # [ 1.298250] ACPI: \_SB_.LNKD: Enabled at IRQ 11 gitlab # [ 1.301857] PCI: CLS 0 bytes, default 64 gitlab # [ 1.303273] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) gitlab # [ 1.303852] Trying to unpack rootfs image as initramfs... gitlab # [ 1.305278] software IO TLB: mapped [mem 0x00000000ba31c000-0x00000000be31c000] (64MB) gitlab # [ 1.305412] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns gitlab # [ 1.355036] Initialise system trusted keyrings gitlab # [ 1.358549] workingset: timestamp_bits=40 max_order=21 bucket_order=0 gitlab # [ 1.381101] Key type asymmetric registered gitlab # [ 1.382532] Asymmetric key parser 'x509' registered gitlab # [ 1.384323] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) gitlab # [ 1.387247] io scheduler mq-deadline registered gitlab # [ 1.388777] io scheduler kyber registered gitlab # [ 1.393085] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled gitlab # [ 1.395526] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A gitlab # [ 1.399927] Linux agpgart interface v0.103 gitlab # [ 1.401310] ACPI: bus type drm_connector registered gitlab # [ 1.405171] usbcore: registered new interface driver usbserial_generic gitlab # [ 1.407145] usbserial: USB Serial support registered for generic gitlab # [ 1.408976] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. gitlab # [ 1.408976] Please enable it if your BIOS has the CPPC option. gitlab # [ 1.413032] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled gitlab # [ 1.415482] drop_monitor: Initializing network drop monitor service gitlab # [ 1.417666] NET: Registered PF_INET6 protocol family gitlab # [ 1.420310] Segment Routing with IPv6 gitlab # [ 1.421537] In-situ OAM (IOAM) with IPv6 gitlab # [ 1.424344] IPI shorthand broadcast: enabled gitlab # [ 1.429474] sched_clock: Marking stable (1160029632, 268590105)->(1504479991, -75860254) gitlab # [ 1.432442] registered taskstats version 1 gitlab # [ 1.434218] Loading compiled-in X.509 certificates gitlab # [ 1.444795] Demotion targets for Node 0: null gitlab # [ 1.446584] Key type .fscrypt registered gitlab # [ 1.447887] Key type fscrypt-provisioning registered gitlab # [ 1.449625] ima: No TPM chip found, activating TPM-bypass! gitlab # [ 1.451342] ima: Allocated hash algorithm: sha1 gitlab # [ 1.452821] ima: No architecture policies found gitlab # [ 1.454764] PM: Magic number: 10:711:495 gitlab # [ 1.457079] RAS: Correctable Errors collector initialized. gitlab # [ 1.462529] clk: Disabling unused clocks gitlab # [ 1.463874] PM: genpd: Disabling unused power domains gitlab # [ 1.606086] Freeing initrd memory: 29392K gitlab # [ 1.610492] Freeing unused decrypted memory: 2028K gitlab # [ 1.614332] Freeing unused kernel image (initmem) memory: 3640K gitlab # [ 1.616305] Write protecting the kernel read-only data: 32768k gitlab # [ 1.619456] Freeing unused kernel image (text/rodata gap) memory: 1220K gitlab # [ 1.622176] Freeing unused kernel image (rodata/data gap) memory: 760K gitlab # [ 1.675687] x86/mm: Checked W+X mappings: passed, no W+X pages found. gitlab # [ 1.677641] Run /init as init process gitlab # [ 1.691307] systemd[1]: Inserted module 'autofs4' gitlab # [ 1.718864] fuse: init (API version 7.45) gitlab # [ 1.728972] ACPI: \_SB_.LNKC: Enabled at IRQ 10 gitlab # [ 1.738359] ACPI: \_SB_.LNKA: Enabled at IRQ 10 gitlab # [ 1.756439] ACPI: \_SB_.LNKB: Enabled at IRQ 11 gitlab # [ 1.804477] systemd[1]: Successfully made /usr/ read-only. gitlab # [ 2.143842] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 2.166085] systemd[1]: Detected virtualization kvm. gitlab # [ 2.170129] systemd[1]: Detected architecture x86-64. gitlab # [ 2.174221] systemd[1]: Running in initrd. gitlab # [ 2.178356] systemd[1]: Initializing machine ID from random generator. gitlab # [ 2.183788] systemd[1]: Hostname set to . gitlab # [ 2.303505] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 2.342554] systemd[1]: Queued start job for default target Initrd Default Target. gitlab # [ 2.359755] systemd[1]: Created slice Slice /system/modprobe. gitlab # [ 2.361847] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 2.364369] systemd[1]: Expecting device /dev/disk/by-label/nix-store... gitlab # [ 2.366467] systemd[1]: Expecting device /dev/disk/by-label/nixos... gitlab # [ 2.368511] systemd[1]: Reached target Path Units. gitlab # [ 2.370155] systemd[1]: Reached target Slice Units. gitlab # [ 2.371764] systemd[1]: Reached target Swaps. gitlab # [ 2.373229] systemd[1]: Reached target Timer Units. gitlab # [ 2.374986] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 2.377133] systemd[1]: Listening on Journal Socket (/dev/log). gitlab # [ 2.379168] systemd[1]: Listening on Journal Sockets. gitlab # [ 2.380947] systemd[1]: Listening on udev Control Socket. gitlab # [ 2.382805] systemd[1]: Listening on udev Kernel Socket. gitlab # [ 2.384545] systemd[1]: Reached target Socket Units. gitlab # [ 2.387440] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 2.390531] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 2.393561] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 2.397518] systemd[1]: Starting Journal Service... gitlab # [ 2.402782] systemd[1]: Starting Load Kernel Modules... gitlab # [ 2.404507] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 2.412963] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 2.422898] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 2.430957] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 2.440762] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 2.448706] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config gitlab # [ 2.459732] systemd-journald[91]: Collecting audit messages is disabled. gitlab # [ 2.461993] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 2.462298] netfs: FS-Cache loaded gitlab # [ 2.465752] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. gitlab # [ 2.471655] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev gitlab # [ 2.478249] 9pnet: Installing 9P2000 support gitlab # [ 2.480496] systemd[1]: Finished Load Kernel Modules. gitlab # [ 2.483294] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 2.487468] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 2.490730] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 2.493119] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 2.496217] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 2.503686] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 2.506766] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 2.508881] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 2.511107] systemd[1]: Reached target Local File Systems. gitlab # [ 2.514004] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 2.532957] systemd[1]: Started Journal Service. gitlab # [ 2.264494] systemd-modules-load[93]: Using 4 probe threads gitlab # [ 2.267843] systemd-modules-load[93]: Inserted module 'virtio_balloon' gitlab # [ 2.270466] systemd-modules-load[93]: Inserted module 'virtio_gpu' gitlab # [ 2.272195] systemd-modules-load[93]: Inserted module 'dm_mod' gitlab # [ 2.274086] systemd-udevd[103]: Using default interface naming scheme 'v261'. gitlab # [ 2.276155] systemd[1]: Starting Create System Files and Directories... gitlab # [ 2.281987] systemd[1]: Finished Create System Files and Directories. gitlab # [ 2.290118] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 2.322694] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 2.324244] systemd[1]: Reached target System Initialization. gitlab # [ 2.325803] systemd[1]: Reached target Basic System. gitlab # [ 2.699878] uhci_hcd 0000:00:01.2: UHCI Host Controller gitlab # [ 2.701251] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1 gitlab # [ 2.703138] uhci_hcd 0000:00:01.2: detected 2 ports gitlab # [ 2.705066] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140 gitlab # [ 2.707555] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 gitlab # [ 2.707810] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 gitlab # [ 2.711506] serio: i8042 KBD port at 0x60,0x64 irq 1 gitlab # [ 2.713157] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 gitlab # [ 2.713233] serio: i8042 AUX port at 0x60,0x64 irq 12 gitlab # [ 2.713572] usb usb1: Product: UHCI Host Controller gitlab # [ 2.714757] SCSI subsystem initialized gitlab # [ 2.715624] usb usb1: Manufacturer: Linux 6.18.43 uhci_hcd gitlab # [ 2.719782] usb usb1: SerialNumber: 0000:00:01.2 gitlab # [ 2.721185] hub 1-0:1.0: USB hub found gitlab # [ 2.722229] hub 1-0:1.0: 2 ports detected gitlab # [ 2.750361] virtio_blk virtio4: 4/0/0 default/read/poll queues gitlab # [ 2.767445] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) gitlab # [ 2.775497] scsi host0: ata_piix gitlab # [ 2.776651] virtio_blk virtio5: 4/0/0 default/read/poll queues gitlab # [ 2.777767] scsi host1: ata_piix gitlab # [ 2.779549] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0 gitlab # [ 2.781815] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0 gitlab # [ 2.783354] virtio_blk virtio5: [vdb] 11843408 512-byte logical blocks (6.06 GB/5.65 GiB) gitlab # [ 2.523534] systemd[1]: Starting Virtual Console Setup... gitlab # [ 2.527137] (udev-worker)[121]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 2.530838] (udev-worker)[121]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.533708] (udev-worker)[134]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.556678] systemd-vconsole-setup[150]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 2.560165] systemd[1]: Finished Virtual Console Setup. gitlab # [ 2.940888] ata2: found unknown device (class 0) gitlab # [ 2.942539] usb 1-1: new full-speed USB device number 2 using uhci_hcd gitlab # [ 2.944248] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 gitlab # [ 2.946478] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 gitlab # [ 2.978212] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 gitlab # [ 2.725491] systemd[1]: Found device /dev/disk/by-label/nixos. gitlab # [ 2.729462] systemd[1]: Found device /dev/disk/by-label/nix-store. gitlab # [ 2.730939] systemd[1]: Reached target Initrd Root Device. gitlab # [ 2.733301] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... gitlab # [ 3.004561] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray gitlab # [ 3.046130] cdrom: Uniform CD-ROM driver Revision: 3.20 gitlab # [ 2.783112] systemd-fsck[159]: nixos: clean, 12/65536 files, 13019/262144 blocks gitlab # [ 2.787858] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. gitlab # [ 3.113035] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 gitlab # [ 3.117930] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 gitlab # [ 3.122524] usb 1-1: Product: QEMU USB Tablet gitlab # [ 3.125454] usb 1-1: Manufacturer: QEMU gitlab # [ 3.128078] usb 1-1: SerialNumber: 28754-0000:00:01.2-1 gitlab # [ 3.146442] hid: raw HID events driver (C) Jiri Kosina gitlab # [ 3.155026] usbcore: registered new interface driver usbhid gitlab # [ 3.156410] usbhid: USB HID core driver gitlab # [ 3.167338] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 gitlab # [ 3.173965] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0 gitlab # [ 3.154194] systemd[1]: Mounting /sysroot... gitlab # [ 3.584829] EXT4-fs (vda): mounted filesystem 1b1fb2e7-0d67-4a09-812a-d7daad8b8b80 r/w with ordered data mode. Quota mode: none. gitlab # [ 3.320755] systemd[1]: Mounted /sysroot. gitlab # [ 3.321997] systemd[1]: Reached target Initrd Root File System. gitlab # [ 3.323737] systemd[1]: Mounting /sysroot/nix/.ro-store... gitlab # [ 3.325227] systemd[1]: Mounting /sysroot/nix/.rw-store... gitlab # [ 3.328388] systemd[1]: Mounting /sysroot/run... gitlab # [ 3.329665] systemd[1]: Mounting /sysroot/tmp/shared... gitlab # [ 3.334280] systemd[1]: Mounting /sysroot/tmp/xchg... gitlab # [ 3.338149] systemd[1]: Starting Mountpoints Configured in the Real Root... gitlab # [ 3.349719] systemd-sysroot-fstab-check[179]: /sysroot should be mounted in the initrd, will request daemon-reload. gitlab # [ 3.352904] systemd[1]: Mounted /sysroot/run. gitlab # [ 3.354668] systemd[1]: Mount[ 3.623312] 9p: Installing v9fs 9p2000 file system support gitlab # ed /sysroot/nix/.rw-store. gitlab # [ 3.627503] erofs (device vdb): mounted with root inode @ nid 128. gitlab # [ 3.360443] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.363665] systemd[1]: Reload requested from client PID 179 ('systemd-sysroot') (unit initrd-parse-etc.service)... gitlab # [ 3.366468] systemd[1]: Reloading... gitlab # [ 3.428852] systemd[1]: Reloading finished in 67 ms. gitlab # [ 3.450397] systemd-sysroot-fstab-check[179]: Requesting initrd-fs.target/start/replace... gitlab # [ 3.454386] systemd[1]: Mounted /sysroot/nix/.ro-store. gitlab # [ 3.456783] systemd[1]: Mounted /sysroot/tmp/shared. gitlab # [ 3.459083] systemd[1]: Mounted /sysroot/tmp/xchg. gitlab # [ 3.461743] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.465986] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.469629] systemd-sysroot-fstab-check[179]: Requesting swap.target/start/replace... gitlab # [ 3.474206] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.477860] systemd[1]: initrd-parse-etc.service: Deactivated successfully. gitlab # [ 3.481848] systemd[1]: Finished Mountpoints Configured in the Real Root. gitlab # [ 3.485390] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. gitlab # [ 3.489284] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.492873] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 4.154830] systemd[1]: Mounting /sysroot/nix/store... gitlab # [ 4.208292] systemd[1]: Mounted /sysroot/nix/store. gitlab # [ 4.211880] systemd[1]: Reached target Initrd File Systems. gitlab # [ 4.215919] systemd[1]: Starting Find NixOS closure... gitlab # [ 4.219682] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... gitlab # [ 4.238122] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. gitlab # [ 4.242782] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. gitlab # [ 4.253885] systemd[1]: Finished Find NixOS closure. gitlab # [ 4.256906] systemd[1]: Reached target Initrd Default Target. gitlab # [ 4.259373] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... gitlab # [ 4.299464] systemd[1]: Stopped target Initrd Default Target. gitlab # [ 4.304120] systemd[1]: Stopped target Basic System. gitlab # [ 4.307421] systemd[1]: Stopped target Initrd Root Device. gitlab # [ 4.310741] systemd[1]: Stopped target Path Units. gitlab # [ 4.313819] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. gitlab # [ 4.318515] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. gitlab # [ 4.322908] systemd[1]: Stopped target Slice Units. gitlab # [ 4.325856] systemd[1]: Stopped target Socket Units. gitlab # [ 4.330071] systemd[1]: Stopped target System Initialization. gitlab # [ 4.333472] systemd[1]: Stopped target Swaps. gitlab # [ 4.335410] systemd[1]: Stopped target Timer Units. gitlab # [ 4.337365] systemd[1]: dbus.socket: Deactivated successfully. gitlab # [ 4.339688] systemd[1]: Closed D-Bus System Message Bus Socket. gitlab # [ 4.342061] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. gitlab # [ 4.344730] systemd[1]: Stopped Find NixOS closure. gitlab # [ 4.346707] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 4.349305] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 4.350746] systemd[1]: systemd-sysctl.service: Deactivated successfully. gitlab # [ 4.352355] systemd[1]: Stopped Apply Kernel Variables. gitlab # [ 4.353720] systemd[1]: systemd-modules-load.service: Deactivated successfully. gitlab # [ 4.355587] systemd[1]: Stopped Load Kernel Modules. gitlab # [ 4.356997] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. gitlab # [ 4.359054] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. gitlab # [ 4.360980] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. gitlab # [ 4.362860] systemd[1]: Stopped Create System Files and Directories. gitlab # [ 4.364687] systemd[1]: Stopped target Local File Systems. gitlab # [ 4.366086] systemd[1]: Stopped target Preparation for Local File Systems. gitlab # [ 4.367706] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. gitlab # [ 4.369403] systemd[1]: Stopped Coldplug All udev Devices. gitlab # [ 4.370745] systemd[1]: Stopping Rule-based Manager for Device Events and Files... gitlab # [ 4.372504] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 4.374209] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 4.375504] systemd[1]: initrd-cleanup.service: Deactivated successfully. gitlab # [ 4.377304] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. gitlab # [ 4.378888] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 4.380604] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 4.381986] systemd[1]: systemd-udevd.service: Deactivated successfully. gitlab # [ 4.383577] systemd[1]: Stopped Rule-based Manager for Device Events and Files. gitlab # [ 4.385330] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. gitlab # [ 4.387066] systemd[1]: Closed udev Control Socket. gitlab # [ 4.388368] systemd[1]: Starting Cleanup udev Database... gitlab # [ 4.389678] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. gitlab # [ 4.391442] systemd[1]: Stopped Create Static Device Nodes in /dev. gitlab # [ 4.392902] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. gitlab # [ 4.394800] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. gitlab # [ 4.396580] systemd[1]: kmod-static-nodes.service: Deactivated successfully. gitlab # [ 4.398242] systemd[1]: Stopped Create List of Static Device Nodes. gitlab # [ 4.399711] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. gitlab # [ 4.401481] systemd[1]: Finished Cleanup udev Database. gitlab # [ 4.402772] systemd[1]: Reached target Switch Root. gitlab # [ 4.404218] systemd[1]: Starting NixOS Activation... gitlab # [ 4.465098] initrd-nixos-activation-start[230]: booting system configuration /nix/store/rq9jh1pxl6pghpfq01jiw9c8hc19zfls-nixos-system-gitlab-test gitlab # [ 4.490373] initrd-nixos-activation-start[230]: running activation script... gitlab # [ 4.699985] initrd-nixos-activation-start[253]: setting up /etc... gitlab # [ 4.826202] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. gitlab # [ 4.828193] systemd[1]: Finished NixOS Activation. gitlab # [ 4.831961] systemd[1]: Starting Switch Root... gitlab # [ 4.873987] systemd[1]: Switching root. gitlab # [ 5.205567] systemd-journald[91]: Received SIGTERM from PID 1 (systemd). gitlab # [ 5.350304] NET: Registered PF_VSOCK protocol family gitlab # [ 5.731557] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 5.748101] systemd[1]: Detected virtualization kvm. gitlab # [ 5.751324] systemd[1]: Detected architecture x86-64. gitlab # [ 5.754892] systemd[1]: Detected first boot. gitlab # [ 5.758954] systemd[1]: Initializing machine ID from random generator. gitlab # [ 5.922751] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 6.019366] systemd[1]: Applying preset policy. gitlab # [ 6.177627] systemd[1]: Populated /etc with preset unit settings. gitlab # [ 6.310000] systemd[1]: initrd-switch-root.service: Deactivated successfully. gitlab # [ 6.312262] systemd[1]: Stopped initrd-switch-root.service. gitlab # [ 6.315460] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. gitlab # [ 6.319063] systemd[1]: Created slice Slice /system/getty. gitlab # [ 6.321196] systemd[1]: Created slice GitLab DevOps Platform Slice. gitlab # [ 6.323366] systemd[1]: Created slice User and Session Slice. gitlab # [ 6.325071] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 6.327122] systemd[1]: Started Forward Password Requests to Wall Directory Watch. gitlab # [ 6.329006] systemd[1]: Expecting device /dev/hvc0... gitlab # [ 6.330272] systemd[1]: Expecting device /dev/ttyS0... gitlab # [ 6.331636] systemd[1]: Reached target Local Encrypted Volumes. gitlab # [ 6.333146] systemd[1]: Stopped target initrd-fs.target. gitlab # [ 6.334489] systemd[1]: Stopped target initrd-root-fs.target. gitlab # [ 6.335976] systemd[1]: Stopped target initrd-switch-root.target. gitlab # [ 6.337533] systemd[1]: Reached target Virtual Machines and Containers. gitlab # [ 6.339204] systemd[1]: Reached target Path Units. gitlab # [ 6.340508] systemd[1]: Reached target Remote File Systems. gitlab # [ 6.342013] systemd[1]: Reached target Slice Units. gitlab # [ 6.343318] systemd[1]: Reached target Swaps. gitlab # [ 6.345414] systemd[1]: Listening on Query the User Interactively for a Password. gitlab # [ 6.348314] systemd[1]: Listening on Process Core Dump Socket. gitlab # [ 6.350428] systemd[1]: Listening on Credential Encryption/Decryption. gitlab # [ 6.352892] systemd[1]: Listening on Factory Reset Management. gitlab # [ 6.354483] systemd[1]: Listening on Hostname Service Socket. gitlab # [ 6.357845] systemd[1]: Starting Journal Log Access Socket... gitlab # [ 6.360136] systemd[1]: Listening on Journal Audit Socket. gitlab # [ 6.362844] systemd[1]: Listening on Console Output Muting Service Socket. gitlab # [ 6.364814] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. gitlab # [ 6.366698] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.368966] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 6.372725] systemd[1]: Listening on Disk Repartitioning Service Socket. gitlab # [ 6.374535] systemd[1]: Listening on udev Control Socket. gitlab # [ 6.376110] systemd[1]: Listening on udev Varlink Socket. gitlab # [ 6.379903] systemd[1]: Mounting Huge Pages File System... gitlab # [ 6.397058] systemd[1]: Mounting POSIX Message Queue File System... gitlab # [ 6.400010] systemd[1]: Mounting Kernel Debug File System... gitlab # [ 6.402915] systemd[1]: Mounting Kernel Trace File System... gitlab # [ 6.416096] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 6.417757] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 6.423948] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 6.427840] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm gitlab # [ 6.434105] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 6.440788] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 6.450059] systemd[1]: Mounting FUSE Control File System... gitlab # [ 6.453915] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 6.471912] systemd[1]: Starting Journal Service... gitlab # [ 6.477245] systemd[1]: Starting Load Kernel Modules... gitlab # [ 6.496367] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... gitlab # [ 6.503327] systemd[1]: Starting Remount Root and Kernel File Systems... gitlab # [ 6.505033] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.508461] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 6.514143] systemd[1]: Listening on Journal Log Access Socket. gitlab # [ 6.518118] systemd[1]: Mounted Huge Pages File System. gitlab # [ 6.519751] systemd[1]: Mounted POSIX Message Queue File System. gitlab # [ 6.521460] systemd[1]: Mounted Kernel Debug File System. gitlab # [ 6.523164] systemd[1]: Mounted Kernel Trace File System. gitlab # [ 6.528366] EXT4-fs (vda): re-mounted 1b1fb2e7-0d67-4a09-812a-d7daad8b8b80. gitlab # [ 6.528442] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 6.530248] systemd-journald[325]: Collecting audit messages is enabled. gitlab # [ 6.531721] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 6.536830] loop: module loaded gitlab # [ 6.537370] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 6.539569] systemd[1]: Mounted FUSE Control File System. gitlab # [ 6.541976] systemd[1]: Finished Load Kernel Modules. gitlab # [ 6.544249] systemd[1]: Finished Remount Root and Kernel File Systems. gitlab # [ 6.547039] systemd[1]: Listening on Disk Image Download Service Socket. gitlab # [ 6.550144] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 6.293496] systemd[1]: Queued start job for default target M[ 6.562742] systemd[1]: Starting Firewall... gitlab # ulti-User System. gitlab # [ 6.564258] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 6.296491] systemd[1]: Unnecessary job was removed for /dev/vdb. gitlab # [ 6.300468] systemd[1]: systemd-journald.service: Deactivated successfully. gitlab # [ 6.302556] systemd-modules-load[326]: Using 3 probe threads gitlab # [ 6.304234] systemd-modules-load[326]: Inserted module 'tls' gitlab # [ 6.573746] systemd[1]: Starting Load/Save OS Random Seed... gitlab # [ 6.305958] systemd-modules-load[326]: Inserted module 'loop' gitlab # [ 6.308779] systemd-oomd[328]: No swap; memory pressure usage will be degraded gitlab # [ 6.578664] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 6.587270] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 6.589393] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.593411] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. gitlab # [ 6.595237] systemd[1]: Started Journal Service. gitlab # [ 6.330254] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 6.334427] systemd[1]: Finished Load/Save OS Random Seed. gitlab # [ 6.337496] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 6.345666] systemd[1]: Reached target First Boot Complete. gitlab # [ 6.348243] systemd[1]: Starting Flush Journal to Persistent Storage... gitlab # [ 6.358789] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 6.360529] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 6.635282] systemd-journald[325]: Received client request to flush runtime journal. gitlab # [ 6.428776] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 6.430877] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 6.432759] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 6.434707] systemd-udevd[374]: Using default interface naming scheme 'v261'. gitlab # [ 6.436439] systemd[1]: Finished Flush Journal to Persistent Storage. gitlab # [ 6.440283] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 6.449636] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 6.561119] systemd[1]: Finished Firewall. gitlab # [ 6.562454] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 6.584211] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. gitlab # [ 6.628742] (udev-worker)[468]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.631709] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. gitlab # [ 6.635747] (udev-worker)[470]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 6.639272] (udev-worker)[470]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.934540] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console gitlab # [ 6.939181] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 gitlab # [ 6.945969] ACPI: button: Power Button [PWRF] gitlab # [ 6.948937] Console: switching to colour dummy device 80x25 gitlab # [ 6.952699] [drm] Found bochs VGA, ID 0xb0c5. gitlab # [ 6.953480] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. gitlab # [ 6.958098] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic gitlab # [ 6.959405] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0 gitlab # [ 6.699942] systemd[1]: Condition check resulted in Virtio network device being skipped. gitlab # [ 6.703963] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 6.706617] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 6.710595] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 6.713489] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.715722] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.965285] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 gitlab # [ 6.965351] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD gitlab # [ 6.987893] Floppy drive(s): fd0 is 2.88M AMI BIOS gitlab # [ 6.988267] Console: switching to colour frame buffer device 160x50 gitlab # [ 6.994803] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device gitlab # [ 6.999208] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 gitlab # [ 7.000983] FDC 0 is a S82078B gitlab # [ 7.001523] rtc_cmos 00:05: RTC can wake from S4 gitlab # [ 7.004302] parport_pc 00:03: reported by Plug and Play ACPI gitlab # [ 7.006255] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 gitlab # [ 7.009500] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] gitlab # [ 7.010097] rtc_cmos 00:05: registered as rtc0 gitlab # [ 7.011968] rtc_cmos 00:05: setting system clock to 2026-09-10T19:29:26 UTC (1789068566) gitlab # [ 7.013728] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs gitlab # [ 7.016091] systemd-journald[325]: Time jumped backwards, rotating. gitlab # [ 7.022123] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input6 gitlab # [ 7.039187] mousedev: PS/2 mouse device common for all mice gitlab # [ 6.825557] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.847270] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 6.850869] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 6.881138] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.888963] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 6.891847] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 7.161664] ppdev: user-space parallel port driver gitlab # [ 6.899046] systemd[1]: Starting Virtual Console Setup... gitlab # [ 7.187752] kvm_amd: TSC scaling supported gitlab # [ 7.188888] kvm_amd: Nested Virtualization enabled gitlab # [ 7.190063] kvm_amd: Nested Paging enabled gitlab # [ 7.191034] kvm_amd: LBR virtualization supported gitlab # [ 7.192267] kvm_amd: Virtual VMLOAD VMSAVE supported gitlab # [ 7.193512] kvm_amd: Virtual GIF supported gitlab # [ 7.194649] kvm_amd: Virtual NMI enabled gitlab # [ 7.242077] EDAC MC: Ver: 3.0.0 gitlab # [ 6.979965] systemd-vconsole-setup[500]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 6.983639] systemd[1]: Finished Virtual Console Setup. gitlab # [ 7.043611] systemd[1]: Mounting /run/wrappers... gitlab # [ 7.065460] systemd[1]: Mounted /run/wrappers. gitlab # [ 7.067810] systemd[1]: Reached target Local File Systems. gitlab # [ 7.069927] systemd[1]: Listening on Boot Loader Control Service Socket. gitlab # [ 7.072806] systemd[1]: Starting register-nix-paths.service... gitlab # [ 7.075552] systemd[1]: Starting Create SUID/SGID Wrappers... gitlab # [ 7.077991] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 7.084599] systemd[1]: Starting Save Transient machine-id to Disk... gitlab # [ 7.087455] systemd[1]: Starting Create System Files and Directories... gitlab # [ 7.122382] systemd[1]: Finished Create System Files and Directories. gitlab # [ 7.124407] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. gitlab # [ 7.126406] systemd[1]: Starting Rebuild Journal Catalog... gitlab # [ 7.127863] systemd[1]: Starting Record System Boot/Shutdown in UTMP... gitlab # [ 7.130172] systemd[1]: Finished Save Transient machine-id to Disk. gitlab # [ 7.142189] systemd[1]: Finished Record System Boot/Shutdown in UTMP. gitlab # [ 7.156945] systemd[1]: Finished Rebuild Journal Catalog. gitlab # [ 7.160236] systemd[1]: Starting Update is Completed... gitlab # [ 7.175049] systemd[1]: Finished Update is Completed. gitlab # [ 7.202236] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. gitlab # [ 7.203963] systemd[1]: Finished Create SUID/SGID Wrappers. gitlab # [ 7.436558] systemd[1]: Finished register-nix-paths.service. gitlab # [ 7.440852] systemd[1]: Reached target System Initialization. gitlab # [ 7.445241] systemd[1]: Started Discard unused filesystem blocks once a week. gitlab # [ 7.449593] systemd[1]: Started logrotate.timer. gitlab # [ 7.452560] systemd[1]: Started Daily Cleanup of Temporary Directories. gitlab # [ 7.456476] systemd[1]: Reached target Timer Units. gitlab # [ 7.459585] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 7.463395] systemd[1]: Listening on Nix Daemon Socket. gitlab # [ 7.476660] systemd[1]: Starting nixbot CI service socket... gitlab # [ 7.480180] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. gitlab # [ 7.485179] systemd[1]: Starting D-Bus System Message Bus... gitlab # [ 7.488652] systemd[1]: Listening on nixbot CI service socket. gitlab # [ 7.492536] systemd[1]: Reached target Socket Units. gitlab # [ 7.541503] dbus-broker-launch[582]: Looking up NSS user entry for 'systemd-timesync'... gitlab # [ 7.546548] dbus-broker-launch[582]: NSS returned no entry for 'systemd-timesync' gitlab # [ 7.550935] dbus-broker-launch[582]: Invalid user-name in /nix/store/37yal9w16hfdxjh64sww8gn81f4hma6i-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" gitlab # [ 7.559648] systemd[1]: Started D-Bus System Message Bus. gitlab # [ 7.562369] dbus-broker-launch[582]: Ready gitlab # [ 7.564370] systemd[1]: Reached target Basic System. gitlab # [ 7.567401] systemd[1]: Started backdoor.service. gitlab # [ 7.570952] systemd[1]: Starting gitlab-config.service... gitlab # [ 7.590147] systemd[1]: Starting Import lastlog data into lastlog2 database... gitlab # [ 7.597479] systemd[1]: Starting Logrotate configuration check... gitlab # [ 7.602848] logrotate[587]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. gitlab # [ 7.610905] logrotate[587]: reading config file /nix/store/y3500wq742i80l631mh8nc8rchpsfb9q-logrotate.conf gitlab # [ 7.615344] logrotate[587]: note: 'monthly' overrides previously specified 'weekly' gitlab # [ 7.618249] logrotate[587]: Reading state from file: /var/lib/logrotate.status gitlab # connecting to host... gitlab # [ 7.623256] logrotate[587]: state file /var/lib/logrotate.status does not exist gitlab # [ 7.625605] logrotate[587]: Allocating hash table for state file, size 64 entries gitlab # [ 7.627592] logrotate[587]: Handling 4 logs gitlab # [ 7.628819] logrotate[587]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.629665] logrotate[587]: considering log /var/log/btmp gitlab # [ 7.631043] logrotate[587]: Creating new state gitlab # [ 7.631750] logrotate[587]: Now: 2026-09-10 19:29 gitlab: Guest shell says: b'Spawning backdoor root shell...\n' gitlab # [ 7.632871] logrotate[587]: Last rotated at 2026-09-10 19:00 gitlab # [ 7.633960] logrotate[587]: log does not need rotating (log has already been rotated) gitlab: connected to guest root shell gitlab # [ 7.636791] logrotate[587]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab: (connecting took 8.56 seconds) gitlab # [ 7.638251] logrotate[587]: considering log /var/log/wtmp gitlab: (finished: waiting for the VM to finish booting, in 8.56 seconds) gitlab # [ 7.639171] logrotate[587]: Creating new state gitlab # [ 7.639942] logrotate[587]: Now: 2026-09-10 19:29 gitlab # [ 7.641082] logrotate[587]: Last rotated at 2026-09-10 19:00 gitlab # [ 7.641819] logrotate[587]: log does not need rotating (log has already been rotated) gitlab # [ 7.642634] logrotate[587]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed gitlab # [ 7.643693] logrotate[587]: switching euid from 0 to 165 and egid from 0 to 165 (pid 587) gitlab # [ 7.645914] logrotate[587]: considering log /var/gitlab/state/log/*.log gitlab # [ 7.646851] logrotate[587]: log /var/gitlab/state/log/*.log does not exist -- skipping gitlab # [ 7.647941] logrotate[587]: Creating new state gitlab # [ 7.649072] logrotate[587]: switching euid from 165 to 0 and egid from 165 to 0 (pid 587) gitlab # [ 7.649890] logrotate[587]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed gitlab # [ 7.651825] logrotate[587]: switching euid from 0 to 60 and egid from 0 to 60 (pid 587) gitlab # [ 7.652634] logrotate[587]: considering log /var/log/nginx/*.log gitlab # [ 7.653752] logrotate[587]: log /var/log/nginx/*.log does not exist -- skipping gitlab # [ 7.657540] logrotate[587]: Creating new state gitlab # [ 7.658365] logrotate[587]: not running postrotate script, since no logs were rotated gitlab # [ 7.659229] logrotate[587]: switching euid from 60 to 0 and egid from 60 to 0 (pid 587) gitlab # [ 7.660169] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.661153] systemd[1]: Starting Post-Boot Actions... gitlab # [ 7.662306] systemd[1]: Started Reset console on configuration changes. gitlab # [ 7.663406] systemd[1]: Starting resolvconf update... gitlab # [ 7.664550] systemd[1]: Finished Logrotate configuration check. gitlab # [ 7.665625] systemd[1]: Finished Post-Boot Actions. gitlab # [ 7.676647] systemd[1]: Finished Import lastlog data into lastlog2 database. gitlab # [ 7.693243] nsncd[590]: Sep 10 19:29:27.449 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.694426] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.695500] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.696290] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.699434] systemd[1]: Starting User Login Management... gitlab # [ 7.707664] systemd[1]: Stopped target Host and Network Name Lookups. gitlab # [ 7.708796] systemd[1]: Stopping Host and Network Name Lookups... gitlab # [ 7.711553] systemd[1]: Stopped target User and Group Name Lookups. gitlab # [ 7.712947] systemd[1]: Stopping User and Group Name Lookups... gitlab # [ 7.714680] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... gitlab # [ 7.716498] systemd[1]: nscd.service: Deactivated successfully. gitlab # [ 7.719087] systemd[1]: Stopped Name Service Cache Daemon (nsncd). gitlab # [ 7.720502] systemd[1]: nscd.service: Consumed 20ms CPU time over 112ms wall clock time, 7.3M memory peak, 5.5M read from disk. gitlab # [ 7.724485] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.738777] systemd[1]: Finished resolvconf update. gitlab # [ 7.740480] systemd[1]: Reached target Preparation for Network. gitlab # [ 7.744468] nsncd[680]: Sep 10 19:29:27.499 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.747778] systemd[1]: Starting DHCP Client... gitlab # [ 7.749090] systemd[1]: Starting Address configuration of eth1... gitlab # [ 7.753673] systemd[1]: Starting Extra networking commands.... gitlab # [ 7.755041] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.758949] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.760223] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.765320] systemd-logind[658]: New seat seat0. gitlab # [ 7.770277] systemd-logind[658]: Watching system buttons on /dev/input/event2 (Power Button) gitlab # [ 7.771627] systemd-logind[658]: Watching system buttons on /dev/input/event5 (QEMU Virtio Keyboard) gitlab # [ 7.776098] systemd-logind[658]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) gitlab # [ 7.777301] systemd[1]: Started User Login Management. gitlab # [ 7.778647] systemd[1]: Starting linger-users.service... gitlab # [ 7.795401] network-addresses-eth1-start[702]: adding address 192.168.1.1/24... done gitlab # [ 7.797669] systemd[1]: linger-users.service: Deactivated successfully. gitlab # [ 7.799441] network-addresses-eth1-start[702]: adding address 2001:db8:1::1/64... done gitlab # [ 7.801422] systemd[1]: Finished linger-users.service. gitlab # [ 7.802964] systemd[1]: Finished Address configuration of eth1. gitlab # [ 7.829764] systemd[1]: Finished Extra networking commands.. gitlab # [ 7.831458] systemd[1]: Reached target Network. gitlab # [ 7.832960] systemd[1]: Starting gitlab-workhorse.service... gitlab # [ 7.836477] systemd[1]: Starting Nginx Web Server... gitlab # [ 7.841460] dhcpcd[715]: dhcpcd-10.3.2 starting gitlab # [ 7.842694] systemd[1]: Starting PostgreSQL Server... gitlab # [ 7.845836] systemd[1]: Starting Redis Server - redis-gitlab... gitlab # [ 7.848163] dhcpcd[774]: dev: loaded udev gitlab # [ 7.849397] systemd[1]: Starting Permit User Sessions... gitlab # [ 8.121628] 8021q: 802.1Q VLAN Support v1.8 gitlab # [ 8.125120] 8021q: adding VLAN 0 to HW filter on device eth1 gitlab # [ 7.870098] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. gitlab # [ 7.875542] systemd[1]: Finished Permit User Sessions. gitlab # [ 7.883194] systemd[1]: Started Getty on tty1. gitlab # [ 7.884283] systemd[1]: Reached target Login Prompts. gitlab # [ 8.214477] cfg80211: Loading compiled-in X.509 certificates for regulatory database gitlab # [ 8.221805] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' gitlab # [ 8.224310] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' gitlab # [ 8.227191] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 gitlab # [ 8.229213] cfg80211: failed to load regulatory.db gitlab # [ 7.978372] redis-server[792]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. gitlab # [ 7.982116] redis-server[792]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo gitlab # [ 7.984526] redis-server[792]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=792, just started gitlab # [ 7.985700] redis-server[792]: Configuration loaded gitlab # [ 7.988165] redis[792]: monotonic clock: POSIX clock_gettime gitlab # [ 7.989407] nginx-pre-start[795]: nginx: the configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf syntax is ok gitlab # [ 7.991438] nginx-pre-start[795]: nginx: configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf test is successful gitlab # [ 7.992963] redis[792]: Running mode=standalone, port=0. gitlab # [ 7.993949] systemd[1]: Started Redis Server - redis-gitlab. gitlab # [ 7.994994] redis[792]: Server initialized gitlab # [ 7.995843] redis[792]: Ready to accept connections unix gitlab # [ 7.996967] redis[792]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. gitlab # [ 8.005392] systemd[1]: Started Nginx Web Server. gitlab # [ 8.010643] postgresql-pre-start[799]: The files belonging to this database system will be owned by user "postgres". gitlab # [ 8.011765] postgresql-pre-start[799]: This user must also own the server process. gitlab # [ 8.013245] postgresql-pre-start[799]: The database cluster will be initialized with locale "en_US.UTF-8". gitlab # [ 8.015868] postgresql-pre-start[799]: The default database encoding has accordingly been set to "UTF8". gitlab # [ 8.018211] postgresql-pre-start[799]: The default text search configuration will be set to "english". gitlab # [ 8.020587] postgresql-pre-start[799]: Data page checksums are enabled. gitlab # [ 8.022380] postgresql-pre-start[799]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok gitlab # [ 8.025249] postgresql-pre-start[799]: creating subdirectories ... ok gitlab # [ 8.026332] postgresql-pre-start[799]: selecting dynamic shared memory implementation ... posix gitlab # [ 8.301565] 8021q: adding VLAN 0 to HW filter on device eth0 gitlab # [ 8.036595] dhcpcd[774]: eth0: waiting for carrier gitlab # [ 8.038536] dhcpcd[774]: eth0: carrier acquired gitlab # [ 8.041294] dhcpcd[774]: DUID 00:01:00:01:32:35:c1:97:52:54:00:12:34:56 gitlab # [ 8.042439] dhcpcd[774]: eth0: IAID 00:12:34:56 gitlab # [ 8.043339] dhcpcd[774]: eth0: adding address fe80::5054:ff:fe12:3456 gitlab # [ 8.058101] postgresql-pre-start[799]: selecting default "max_connections" ... 100 gitlab # [ 8.103452] postgresql-pre-start[799]: selecting default "shared_buffers" ... 128MB gitlab # [ 8.314239] systemd[1]: Started gitlab-workhorse.service. gitlab # [ 8.506407] postgresql-pre-start[799]: selecting default time zone ... UTC gitlab # [ 8.509791] postgresql-pre-start[799]: creating configuration files ... ok gitlab # [ 8.522302] gitlab-workhorse[830]: time="2026-09-10T19:29:28Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4 gitlab # [ 8.524085] gitlab-workhorse[830]: time="2026-09-10T19:29:28Z" level=info msg="Using redis/go-redis" gitlab # [ 8.525859] gitlab-workhorse[830]: time="2026-09-10T19:29:28Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix gitlab # [ 8.693803] postgresql-pre-start[799]: running bootstrap script ... ok gitlab # [ 8.832834] dhcpcd[774]: eth0: soliciting a DHCP lease gitlab # [ 9.106522] NET: Registered PF_PACKET protocol family gitlab # [ 8.851618] dhcpcd[774]: eth0: offered 10.0.2.15 from 10.0.2.2 gitlab # [ 8.858360] dhcpcd[774]: eth0: probing address 10.0.2.15/24 gitlab # [ 9.156372] postgresql-pre-start[799]: performing post-bootstrap initialization ... ok gitlab # [ 9.454414] postgresql-pre-start[799]: syncing data to disk ... ok gitlab # [ 9.457864] postgresql-pre-start[799]: initdb: warning: enabling "trust" authentication for local connections gitlab # [ 9.461178] postgresql-pre-start[799]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. gitlab # [ 9.464119] postgresql-pre-start[799]: Success. You can now start the database server using: gitlab # [ 9.467157] postgresql-pre-start[799]: pg_ctl -D /var/lib/postgresql/18 -l logfile start gitlab # [ 9.605308] postgres[855]: [855] LOG: starting PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit gitlab # [ 9.612405] postgres[855]: [855] LOG: listening on IPv6 address "::1", port 5432 gitlab # [ 9.615100] postgres[855]: [855] LOG: listening on IPv4 address "127.0.0.1", port 5432 gitlab # [ 9.624496] postgres[855]: [855] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" gitlab # [ 9.645114] postgres[865]: [865] LOG: database system was shut down at 2026-09-10 19:29:28 GMT gitlab # [ 9.657674] postgres[855]: [855] LOG: database system is ready to accept connections gitlab # [ 9.666852] systemd[1]: Started PostgreSQL Server. gitlab # [ 9.674442] systemd[1]: Starting PostgreSQL Setup Scripts... gitlab # [ 9.892249] postgresql-setup-start[876]: CREATE DATABASE gitlab # [ 9.918170] postgresql-setup-start[890]: CREATE ROLE gitlab # [ 9.925956] postgresql-setup-start[892]: ALTER DATABASE gitlab # [ 9.932409] dhcpcd[774]: eth0: soliciting an IPv6 router gitlab # [ 9.933496] dhcpcd[774]: eth0: Router Advertisement from fe80::2 gitlab # [ 9.934901] dhcpcd[774]: eth0: adding address fec0::5054:ff:fe12:3456/64 gitlab # [ 9.935971] dhcpcd[774]: eth0: adding route to fec0::/64 gitlab # [ 9.937408] dhcpcd[774]: eth0: adding default route via fe80::2 gitlab # [ 9.949377] postgresql-setup-start[897]: CREATE ROLE gitlab # [ 9.951873] systemd[1]: Finished PostgreSQL Setup Scripts. gitlab # [ 9.953526] systemd[1]: Reached target PostgreSQL. gitlab # [ 9.955494] systemd[1]: Starting gitlab-postgresql.service... gitlab # [ 10.058322] gitlab-postgresql-start[905]: CREATE DATABASE gitlab # [ 10.095850] gitlab-postgresql-start[914]: CREATE EXTENSION gitlab # [ 10.135776] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[910]: mkdir -p /var/gitlab/state/home/.ssh: OK gitlab # [ 10.138545] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[910]: chmod 700 /var/gitlab/state/home/.ssh: OK gitlab # [ 10.184612] gitlab-postgresql-start[916]: CREATE EXTENSION gitlab # [ 10.187000] systemd[1]: Finished gitlab-postgresql.service. gitlab # [ 10.193101] systemd[1]: Finished gitlab-config.service. gitlab # [ 10.197153] systemd[1]: Started gitaly.service. gitlab # [ 10.199350] systemd[1]: Starting gitlab-db-config.service... gitlab # [ 10.771686] gitaly[931]: time="2026-09-10T19:29:30.527Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/ymnlr20kjfmhkzg7i9hp25h3ii8qnn0d-gitaly.toml' and fix the errors that are printed." pid=931 gitlab # [ 10.776472] gitaly[931]: time="2026-09-10T19:29:30.527Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=931 gitlab # [ 10.778303] gitaly[931]: time="2026-09-10T19:29:30.527Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=931 gitlab # [ 10.782293] gitaly[931]: time="2026-09-10T19:29:30.527Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=931 gitlab # [ 10.783980] gitaly[931]: time="2026-09-10T19:29:30.527Z" level=info msg="Starting Gitaly" pid=931 version= gitlab # [ 10.978133] gitaly[931]: time="2026-09-10T19:29:30.734Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=205 pid=931 gitlab # [ 10.981570] gitaly[931]: time="2026-09-10T19:29:30.734Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=931 gitlab # [ 10.984768] gitaly[931]: time="2026-09-10T19:29:30.735Z" level=info msg="finished initializing command factory" duration_ms=1 pid=931 gitlab # [ 10.988109] gitaly[931]: time="2026-09-10T19:29:30.735Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-931/git-exec-1156310712.d/git pid=931 gitlab # [ 10.991882] gitaly[931]: time="2026-09-10T19:29:30.740Z" level=info msg="finished detecting git version" duration_ms=4 pid=931 gitlab # [ 10.994954] gitaly[931]: time="2026-09-10T19:29:30.740Z" level=info msg="using Git version" pid=931 version=2.53.g gitlab # [ 10.997836] gitaly[931]: time="2026-09-10T19:29:30.741Z" level=info msg="clearing disk cache object folder" pid=931 storage=default gitlab # [ 11.001075] gitaly[931]: time="2026-09-10T19:29:30.741Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1291251007 pid=931 storage=default gitlab # [ 11.005876] gitaly[931]: time="2026-09-10T19:29:30.741Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1291251007 pid=931 storage=default gitlab # [ 11.011326] gitaly[931]: time="2026-09-10T19:29:30.741Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=931 gitlab # [ 11.016289] gitaly[931]: time="2026-09-10T19:29:30.741Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=931 gitlab # [ 11.021324] gitaly[931]: time="2026-09-10T19:29:30.742Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1291251007 clear_duration_ms=0 pid=931 storage=default gitlab # [ 11.052983] gitaly[931]: time="2026-09-10T19:29:30.809Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=931 gitlab # [ 11.057773] gitaly[931]: time="2026-09-10T19:29:30.809Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=931 schema=unix gitlab # [ 11.058941] gitaly[931]: time="2026-09-10T19:29:30.809Z" level=info msg="listening at address" address=/run/gitaly/gitaly-931/sock.d/intern pid=931 schema=unix gitlab # [ 11.060046] gitaly[931]: time="2026-09-10T19:29:30.809Z" level=info msg="counting repositories" pid=931 gitlab # [ 11.061900] gitaly[931]: time="2026-09-10T19:29:30.810Z" level=info msg="starting to count repositories" pid=931 storage_path=/var/gitlab/state/repositories gitlab # [ 11.063377] gitaly[931]: time="2026-09-10T19:29:30.810Z" level=info msg="finished tempdir cleaner walk" pid=931 storage=default time_ms=0 gitlab # [ 11.065167] gitaly[931]: time="2026-09-10T19:29:30.810Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=931 gitlab # [ 11.065768] gitaly[931]: time="2026-09-10T19:29:30.810Z" level=info msg="Started Gitaly" duration_ms=282 pid=931 gitlab # [ 11.067173] gitaly[931]: time="2026-09-10T19:29:30.810Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=931 gitlab # [ 11.067914] gitaly[931]: time="2026-09-10T19:29:30.810Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=931 storage_path=/var/gitlab/state/repositories gitlab # [ 11.068942] gitaly[931]: time="2026-09-10T19:29:30.810Z" level=info msg="completed counting all repositories" counting_duration=0 pid=931 gitlab # [ 11.070358] gitaly[931]: time="2026-09-10T19:29:30.811Z" level=info msg="maintenance: daily scheduled" pid=931 scheduled="2026-09-11 12:00:00 +0000 UTC" gitlab # [ 11.373766] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[935]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 11.386200] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[935]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 11.398777] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[935]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 13.024633] gitaly[931]: time="2026-09-10T19:29:32.780Z" level=info msg="License database preloaded" duration_ms=2252 pid=931 gitlab # [ 13.882158] dhcpcd[774]: eth0: leased 10.0.2.15 for 86400 seconds gitlab # [ 13.885895] dhcpcd[774]: eth0: adding route to 10.0.2.0/24 gitlab # [ 13.889216] dhcpcd[774]: eth0: adding default route via 10.0.2.2 gitlab # [ 14.028760] systemd[1]: Started DHCP Client. gitlab # [ 14.032087] systemd[1]: Reached target Network is Online. gitlab # [ 41.971839] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[935]: W, [2026-09-10T19:30:01.727425 #935] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 44.204882] postgres[1325]: [1325] ERROR: relation "feature_gates" does not exist at character 523 gitlab # [ 44.208900] postgres[1325]: [1325] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), gitlab # [ 44.212104] postgres[1325]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, gitlab # [ 44.216201] postgres[1325]: c.collname, col_description(a.attrelid, a.attnum) AS comment, gitlab # [ 44.220372] postgres[1325]: attidentity AS identity, gitlab # [ 44.225273] postgres[1325]: attgenerated as attgenerated gitlab # [ 44.228757] postgres[1325]: FROM pg_attribute a gitlab # [ 44.232436] postgres[1325]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum gitlab # [ 44.237362] postgres[1325]: LEFT JOIN pg_type t ON a.atttypid = t.oid gitlab # [ 44.241350] postgres[1325]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation gitlab # [ 44.245885] postgres[1325]: WHERE a.attrelid = '"feature_gates"'::regclass gitlab # [ 44.250806] postgres[1325]: AND a.attnum > 0 AND NOT a.attisdropped gitlab # [ 44.254644] postgres[1325]: ORDER BY a.attnum gitlab # [ 44.257557] postgres[1325]: gitlab # [ 47.736768] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1358]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping gitlab # [ 47.746435] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1358]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping gitlab # [ 48.583347] gitlab-workhorse[830]: time="2026-09-10T19:30:08Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 58.584545] gitlab-workhorse[830]: time="2026-09-10T19:30:18Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 65.749993] postgres[1452]: [1452] ERROR: relation "table_without_models" does not exist gitlab # [ 65.751908] postgres[1452]: [1452] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 68.594784] gitlab-workhorse[830]: time="2026-09-10T19:30:28Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 78.605660] gitlab-workhorse[830]: time="2026-09-10T19:30:38Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 79.756400] systemd[1]: Finished gitlab-db-config.service. gitlab # [ 79.764726] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 79.767997] systemd[1]: Starting gitlab.service... gitlab # [ 80.167570] bundle[1646]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 80.176299] bundle[1646]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 80.184719] bundle[1646]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 82.954760] sidekiq-cluster[1645]: {"severity":"INFO","time":"2026-09-10T19:30:42.693Z","message":"Starting cluster with 1 processes"} gitlab # [ 83.102792] bundle[1646]: {"timestamp":"2026-09-10T19:30:42.832Z","pid":1646,"message":"* Enabling systemd notification integration"} gitlab # [ 83.109766] bundle[1646]: {"timestamp":"2026-09-10T19:30:42.858Z","pid":1646,"message":"Puma starting in cluster mode..."} gitlab # [ 83.116431] bundle[1646]: {"timestamp":"2026-09-10T19:30:42.858Z","pid":1646,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"} gitlab # [ 83.124136] bundle[1646]: {"timestamp":"2026-09-10T19:30:42.858Z","pid":1646,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 83.132447] bundle[1646]: {"timestamp":"2026-09-10T19:30:42.858Z","pid":1646,"message":"* Min threads: 0"} gitlab # [ 83.138265] bundle[1646]: {"timestamp":"2026-09-10T19:30:42.858Z","pid":1646,"message":"* Max threads: 4"} gitlab # [ 83.144429] bundle[1646]: {"timestamp":"2026-09-10T19:30:42.858Z","pid":1646,"message":"* Environment: production"} gitlab # [ 83.150651] bundle[1646]: {"timestamp":"2026-09-10T19:30:42.858Z","pid":1646,"message":"* Master PID: 1646"} gitlab # [ 83.156410] bundle[1646]: {"timestamp":"2026-09-10T19:30:42.858Z","pid":1646,"message":"* Workers: 2"} gitlab # [ 83.160773] bundle[1646]: {"timestamp":"2026-09-10T19:30:42.859Z","pid":1646,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} gitlab # [ 83.166527] bundle[1646]: {"timestamp":"2026-09-10T19:30:42.859Z","pid":1646,"message":"* Preloading application"} gitlab # [ 88.615467] gitlab-workhorse[830]: time="2026-09-10T19:30:48Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 98.625923] gitlab-workhorse[830]: time="2026-09-10T19:30:58Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 106.534624] bundle[1646]: W, [2026-09-10T19:31:06.290181 #1646] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 108.635455] gitlab-workhorse[830]: time="2026-09-10T19:31:08Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 109.417241] sidekiq-cluster[1676]: 2026-09-10T19:31:09.172Z pid=1676 tid=wf8 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1676, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>437444608} gitlab # [ 109.615236] sidekiq-cluster[1676]: W, [2026-09-10T19:31:09.371397 #1676] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 109.697702] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:09.453Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 111.844367] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.599Z","message":"GitLab reliable fetch activated!"} gitlab # [ 111.851417] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.599Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 111.869218] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.625Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} gitlab # [ 111.879831] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.627Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"} gitlab # [ 111.889489] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.630Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} gitlab # [ 111.898522] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.634Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} gitlab # [ 111.907194] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.637Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} gitlab # [ 111.914469] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.640Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} gitlab # [ 111.919721] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.642Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} gitlab # [ 111.923398] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.645Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} gitlab # [ 111.926855] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.647Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} gitlab # [ 111.930352] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.650Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} gitlab # [ 111.934236] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.652Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} gitlab # [ 111.937948] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.655Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} gitlab # [ 111.941408] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.658Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} gitlab # [ 111.945140] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.661Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} gitlab # [ 111.948780] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.663Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} gitlab # [ 111.952634] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.666Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} gitlab # [ 111.956133] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.669Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} gitlab # [ 111.959535] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.672Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} gitlab # [ 111.963266] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.675Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} gitlab # [ 111.967052] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.679Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} gitlab # [ 111.970960] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.682Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} gitlab # [ 111.975091] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.685Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} gitlab # [ 111.978966] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.687Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} gitlab # [ 111.980500] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.690Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} gitlab # [ 111.981911] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.693Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} gitlab # [ 111.983382] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.696Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} gitlab # [ 111.984833] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.699Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} gitlab # [ 111.986301] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.702Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} gitlab # [ 111.987582] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.705Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} gitlab # [ 111.989346] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.729Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} gitlab # [ 111.990893] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.731Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} gitlab # [ 111.992392] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.734Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} gitlab # [ 111.993777] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.738Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} gitlab # [ 111.995289] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.741Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} gitlab # [ 111.996622] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.744Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} gitlab # [ 111.998237] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.747Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} gitlab # [ 111.999822] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.750Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} gitlab # [ 112.001236] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.753Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} gitlab # [ 112.002050] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.756Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} gitlab # [ 112.002905] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.758Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} gitlab # [ 112.005380] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.761Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} gitlab # [ 112.007727] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.764Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} gitlab # [ 112.010756] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.767Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} gitlab # [ 112.013301] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.769Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} gitlab # [ 112.016654] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.773Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} gitlab # [ 112.019789] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.776Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} gitlab # [ 112.022235] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.778Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} gitlab # [ 112.024989] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.781Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} gitlab # [ 112.028146] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.784Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} gitlab # [ 112.031094] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.787Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} gitlab # [ 112.033850] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.790Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} gitlab # [ 112.037113] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.793Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} gitlab # [ 112.047472] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.803Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} gitlab # [ 112.050536] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.807Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} gitlab # [ 112.053478] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.810Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} gitlab # [ 112.056449] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.812Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} gitlab # [ 112.059480] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.816Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} gitlab # [ 112.062321] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.818Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} gitlab # [ 112.064936] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.821Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} gitlab # [ 112.067470] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.824Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} gitlab # [ 112.070272] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.826Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} gitlab # [ 112.072756] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.829Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} gitlab # [ 112.076080] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.832Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} gitlab # [ 112.078920] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.835Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} gitlab # [ 112.081928] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.838Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} gitlab # [ 112.084940] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.841Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} gitlab # [ 112.087568] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.844Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} gitlab # [ 112.089968] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.846Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} gitlab # [ 112.092352] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.848Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} gitlab # [ 112.095205] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.851Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} gitlab # [ 112.097574] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.854Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} gitlab # [ 112.099984] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.856Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} gitlab # [ 112.102599] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.859Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} gitlab # [ 112.104962] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.861Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} gitlab # [ 112.107601] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.864Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} gitlab # [ 112.110066] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.866Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} gitlab # [ 112.112674] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.869Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} gitlab # [ 112.115276] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.871Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} gitlab # [ 112.117707] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.874Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} gitlab # [ 112.120232] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.876Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} gitlab # [ 112.122652] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.879Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} gitlab # [ 112.125107] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.881Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} gitlab # [ 112.128452] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.884Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} gitlab # [ 112.130935] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.887Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} gitlab # [ 112.134148] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.890Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} gitlab # [ 112.136634] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.893Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} gitlab # [ 112.139516] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.896Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} gitlab # [ 112.141982] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.898Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} gitlab # [ 112.144549] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.901Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} gitlab # [ 112.147094] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.903Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} gitlab # [ 112.149548] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.906Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} gitlab # [ 112.151986] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.908Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} gitlab # [ 112.154576] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.911Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} gitlab # [ 112.157322] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.913Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} gitlab # [ 112.160424] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.916Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} gitlab # [ 112.164872] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.920Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} gitlab # [ 112.168259] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.922Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} gitlab # [ 112.171905] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.924Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} gitlab # [ 112.177817] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.934Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} gitlab # [ 112.180446] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.936Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} gitlab # [ 112.182954] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.939Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} gitlab # [ 112.186519] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.943Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} gitlab # [ 112.189614] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.946Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} gitlab # [ 112.192748] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.949Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} gitlab # [ 112.196414] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.952Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} gitlab # [ 112.200309] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.954Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} gitlab # [ 112.203948] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.957Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} gitlab # [ 112.271307] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.960Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} gitlab # [ 112.272181] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:11.962Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} gitlab # [ 118.636907] gitlab-workhorse[830]: time="2026-09-10T19:31:18Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 128.647551] gitlab-workhorse[830]: time="2026-09-10T19:31:28Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 133.361579] postgres[2152]: [2152] ERROR: relation "table_without_models" does not exist gitlab # [ 133.364285] postgres[2152]: [2152] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 133.435550] postgres[2151]: [2151] ERROR: relation "table_without_models" does not exist gitlab # [ 133.437088] postgres[2151]: [2151] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 136.144989] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:35.897Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 136.152620] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:35.903Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 136.156475] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:35.903Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 136.159396] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:35.903Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 136.176253] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:35.932Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 136.368279] bundle[1646]: {"timestamp":"2026-09-10T19:31:36.123Z","pid":1646,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} gitlab # [ 136.369891] bundle[1646]: {"timestamp":"2026-09-10T19:31:36.123Z","pid":1646,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} gitlab # [ 136.371515] bundle[1646]: {"timestamp":"2026-09-10T19:31:36.124Z","pid":1646,"message":"! #\u003cThread:0x000074180f0f41c8 /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} gitlab # [ 136.373216] bundle[1646]: {"timestamp":"2026-09-10T19:31:36.124Z","pid":1646,"message":"! #\u003cThread:0x000074180dfa3310 /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} gitlab # [ 136.374542] bundle[1646]: {"timestamp":"2026-09-10T19:31:36.124Z","pid":1646,"message":"Use Ctrl-C to stop"} gitlab # [ 136.376152] bundle[1646]: {"timestamp":"2026-09-10T19:31:36.124Z","pid":1646,"message":"Sending status to systemd every 1.0 sec"} gitlab # [ 136.867561] bundle[1646]: {"timestamp":"2026-09-10T19:31:36.623Z","pid":1646,"message":"- Worker 0 (PID: 2200) booted in 0.42s, phase: 0"} gitlab # [ 136.920412] bundle[1646]: {"timestamp":"2026-09-10T19:31:36.676Z","pid":1646,"message":"- Worker 1 (PID: 2204) booted in 0.45s, phase: 0"} gitlab # [ 136.929640] systemd[1]: Started gitlab.service. gitlab # [ 136.932672] systemd[1]: Reached target Common target for all GitLab services.. gitlab # [ 136.941641] systemd[1]: Starting setup-gitlab.service... gitlab # [ 136.971352] setup-gitlab-start[2228]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' gitlab # [ 137.639556] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:37.394Z","message":"Cleaning working queues"} gitlab # [ 137.646351] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:37.402Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"6f3da67d33b0d72c2610ccd8","created_at":"2026-09-10T19:30:39.084Z","trace_propagation_headers":{"sentry-trace":"0afa5c28e95e451c9e294802aa19e386-e67ad6a03bef4730","baggage":"sentry-trace_id=0afa5c28e95e451c9e294802aa19e386,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"2a0e643787bef7e473d3e1d4a2f7b229","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-10T19:30:39.094Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-6f3da67d33b0d72c2610ccd8: start","job_status":"start","queue_duration_s":58.307852,"scheduling_latency_s":58.307908} gitlab # [ 137.879789] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:31:37.635Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"6f3da67d33b0d72c2610ccd8","created_at":"2026-09-10T19:30:39.084Z","trace_propagation_headers":{"sentry-trace":"0afa5c28e95e451c9e294802aa19e386-e67ad6a03bef4730","baggage":"sentry-trace_id=0afa5c28e95e451c9e294802aa19e386,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"2a0e643787bef7e473d3e1d4a2f7b229","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-10T19:30:39.094Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-6f3da67d33b0d72c2610ccd8: done: 0.233347 sec","job_status":"done","queue_duration_s":58.307852,"scheduling_latency_s":58.307908,"redis_calls":26,"redis_duration_s":0.012038,"redis_read_bytes":914,"redis_write_bytes":2887,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.006039,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003898,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002101,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.10646,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.12209819999999999,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.233347,"completed_at":"2026-09-10T19:31:37.635Z","load_balancing_strategy":"primary","db_duration_s":0.066694,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 139.293799] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:31:39 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 2295 gitlab # [ 139.301905] setup-gitlab-start[2228]: + TOKEN=nixbot-test-token-of-20-chars gitlab # [ 139.305331] setup-gitlab-start[2228]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n' gitlab # [ 139.341804] runuser[2257]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) gitlab # [ 139.735712] setup-gitlab-start[2258]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 139.744579] setup-gitlab-start[2258]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 139.753550] setup-gitlab-start[2258]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 140.371844] setup-gitlab-start[2258]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 140.380766] setup-gitlab-start[2258]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 140.389417] setup-gitlab-start[2258]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 164.482502] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:04.237Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"5cdcf197d90b46fb6f4a33a4","created_at":"2026-09-10T19:32:04.200Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5f71e50bf37bfbe3db521287936a4d0a","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.235Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-5cdcf197d90b46fb6f4a33a4: start","job_status":"start","queue_duration_s":0.002547,"scheduling_latency_s":0.002598} gitlab # [ 164.634572] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:04.390Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"5cdcf197d90b46fb6f4a33a4","created_at":"2026-09-10T19:32:04.200Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5f71e50bf37bfbe3db521287936a4d0a","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.235Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-5cdcf197d90b46fb6f4a33a4: done: 0.15259 sec","job_status":"done","queue_duration_s":0.002547,"scheduling_latency_s":0.002598,"redis_calls":12,"redis_duration_s":0.017655,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001725,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.011027,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004903,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029844,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00176,"exclusive_lock_hold_duration_s":0.017673755,"duration_s":0.15259,"completed_at":"2026-09-10T19:32:04.390Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013534,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.723919] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:04.394Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"72b39a0264d27021abf81356","created_at":"2026-09-10T19:32:04.316Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f3532782afb44fc7cf1f781b80be1aae","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.342Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-72b39a0264d27021abf81356: start","job_status":"start","queue_duration_s":0.05194,"scheduling_latency_s":0.051992} gitlab # [ 165.304678] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.060Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"72b39a0264d27021abf81356","created_at":"2026-09-10T19:32:04.316Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f3532782afb44fc7cf1f781b80be1aae","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.342Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-72b39a0264d27021abf81356: done: 0.666138 sec","job_status":"done","queue_duration_s":0.05194,"scheduling_latency_s":0.051992,"redis_calls":14,"redis_duration_s":0.013176,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002576,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.008385,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002215,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.174,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.174,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.219391,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001339835,"exclusive_lock_hold_duration_s":0.5675112339999999,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.666138,"completed_at":"2026-09-10T19:32:05.060Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.376062,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.307592] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.062Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"bc6502f715669c3f99d1fa5b","created_at":"2026-09-10T19:32:04.368Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"80472d24b7dd4ff8ee252d11734d3e44","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.418Z","job_size_bytes":3,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-bc6502f715669c3f99d1fa5b: start","job_status":"start","queue_duration_s":0.644099,"scheduling_latency_s":0.644122} gitlab # [ 165.328365] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.081Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"bc6502f715669c3f99d1fa5b","created_at":"2026-09-10T19:32:04.368Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"80472d24b7dd4ff8ee252d11734d3e44","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.418Z","job_size_bytes":3,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-bc6502f715669c3f99d1fa5b: done: 0.018571 sec","job_status":"done","queue_duration_s":0.644099,"scheduling_latency_s":0.644122,"redis_calls":10,"redis_duration_s":0.0024850000000000002,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000453,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001538,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000494,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010886,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000599797,"duration_s":0.018571,"completed_at":"2026-09-10T19:32:05.081Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000444,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.398747] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.082Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"d7661956b2f6d37b1ecc62fa","created_at":"2026-09-10T19:32:04.441Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a5c0922768d1c4911761c99b0a470b39","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.453Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-d7661956b2f6d37b1ecc62fa: start","job_status":"start","queue_duration_s":0.629011,"scheduling_latency_s":0.629032} gitlab # [ 165.416278] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.114Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"d7661956b2f6d37b1ecc62fa","created_at":"2026-09-10T19:32:04.441Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a5c0922768d1c4911761c99b0a470b39","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.453Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-d7661956b2f6d37b1ecc62fa: done: 0.031599 sec","job_status":"done","queue_duration_s":0.629011,"scheduling_latency_s":0.629032,"redis_calls":16,"redis_duration_s":0.003389,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001757,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001208,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000424,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01838,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000356191,"duration_s":0.031599,"completed_at":"2026-09-10T19:32:05.114Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006524,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.522353] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.115Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"081f814994a1c854a45f4a65","created_at":"2026-09-10T19:32:04.501Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"495a7931016de1203e9bb539352c6bb2","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.518Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-081f814994a1c854a45f4a65: start","job_status":"start","queue_duration_s":0.597033,"scheduling_latency_s":0.597054} gitlab # [ 165.523385] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.208Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"081f814994a1c854a45f4a65","created_at":"2026-09-10T19:32:04.501Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"495a7931016de1203e9bb539352c6bb2","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.518Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-081f814994a1c854a45f4a65: done: 0.092837 sec","job_status":"done","queue_duration_s":0.597033,"scheduling_latency_s":0.597054,"redis_calls":10,"redis_duration_s":0.002373,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000415,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001563,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000395,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013929,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000388038,"duration_s":0.092837,"completed_at":"2026-09-10T19:32:05.208Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002449,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.602072] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.210Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"0573c10b8c2df8544e77a6c8","created_at":"2026-09-10T19:32:04.524Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b74d59acba56ecd57a6dcad458fb8900","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.530Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-0573c10b8c2df8544e77a6c8: start","job_status":"start","queue_duration_s":0.680111,"scheduling_latency_s":0.680141} gitlab # [ 165.603480] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.297Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"0573c10b8c2df8544e77a6c8","created_at":"2026-09-10T19:32:04.524Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b74d59acba56ecd57a6dcad458fb8900","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.530Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-0573c10b8c2df8544e77a6c8: done: 0.087162 sec","job_status":"done","queue_duration_s":0.680111,"scheduling_latency_s":0.680141,"redis_calls":14,"redis_duration_s":0.003278,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00091,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001311,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001057,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.055,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.054,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023721,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000579962,"exclusive_lock_hold_duration_s":0.06380503600000001,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.087162,"completed_at":"2026-09-10T19:32:05.297Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.056043,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.680801] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.299Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"978d934de24f23cd24ba6063","created_at":"2026-09-10T19:32:04.534Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"026b6b5cf5281689f64452383319d02f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-10T19:32:04.540Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-978d934de24f23cd24ba6063: start","job_status":"start","queue_duration_s":0.759031,"scheduling_latency_s":0.759053} gitlab # [ 165.682096] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.311Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 165.750310] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.318Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"978d934de24f23cd24ba6063","created_at":"2026-09-10T19:32:04.534Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"026b6b5cf5281689f64452383319d02f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-10T19:32:04.540Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-978d934de24f23cd24ba6063: done: 0.018664 sec","job_status":"done","queue_duration_s":0.759031,"scheduling_latency_s":0.759053,"redis_calls":10,"redis_duration_s":0.002088,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000431,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001268,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000389,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010566,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000340826,"duration_s":0.018664,"completed_at":"2026-09-10T19:32:05.318Z","load_balancing_strategy":"primary","db_duration_s":0.000453,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.752192] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.319Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"6d5c7dc9f11c0839c9d25d2b","created_at":"2026-09-10T19:32:04.550Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"347dc3737f0d359597fe3ee9a2c4a1cf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-10T19:32:04.556Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-6d5c7dc9f11c0839c9d25d2b: start","job_status":"start","queue_duration_s":0.763236,"scheduling_latency_s":0.763256} gitlab # [ 165.820806] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.345Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"6d5c7dc9f11c0839c9d25d2b","created_at":"2026-09-10T19:32:04.550Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"347dc3737f0d359597fe3ee9a2c4a1cf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-10T19:32:04.556Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-6d5c7dc9f11c0839c9d25d2b: done: 0.025737 sec","job_status":"done","queue_duration_s":0.763236,"scheduling_latency_s":0.763256,"redis_calls":10,"redis_duration_s":0.002077,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000385,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001257,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000435,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016763,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000353677,"duration_s":0.025737,"completed_at":"2026-09-10T19:32:05.345Z","load_balancing_strategy":"primary","db_duration_s":0.002357,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.892079] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.346Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"b12c05ad9e7261d65efe5c5d","created_at":"2026-09-10T19:32:04.560Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"559d01987c690600c439a1c13a7be9ed","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.567Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-b12c05ad9e7261d65efe5c5d: start","job_status":"start","queue_duration_s":0.778699,"scheduling_latency_s":0.778721} gitlab # [ 165.893122] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.357Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 165.894078] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.363Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"b12c05ad9e7261d65efe5c5d","created_at":"2026-09-10T19:32:04.560Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"559d01987c690600c439a1c13a7be9ed","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.567Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-b12c05ad9e7261d65efe5c5d: done: 0.017341 sec","job_status":"done","queue_duration_s":0.778699,"scheduling_latency_s":0.778721,"redis_calls":10,"redis_duration_s":0.002091,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000426,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00122,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000445,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010611,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000353956,"duration_s":0.017341,"completed_at":"2026-09-10T19:32:05.363Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000382,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.977232] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.365Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"51d9ffc6fff83703c0dd4a6e","created_at":"2026-09-10T19:32:04.573Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.579Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-51d9ffc6fff83703c0dd4a6e: start","job_status":"start","queue_duration_s":0.786148,"scheduling_latency_s":0.786169} gitlab # [ 165.978797] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.454Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"51d9ffc6fff83703c0dd4a6e","created_at":"2026-09-10T19:32:04.573Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.579Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-51d9ffc6fff83703c0dd4a6e: done: 0.089024 sec","job_status":"done","queue_duration_s":0.786148,"scheduling_latency_s":0.786169,"redis_calls":65,"redis_duration_s":0.01381,"redis_read_bytes":114,"redis_write_bytes":41282,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000772,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":656,"redis_queues_calls":2,"redis_queues_duration_s":0.000776,"redis_queues_read_bytes":3,"redis_queues_write_bytes":36985,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.011678,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3374,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000584,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":267,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.073902,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000396977,"duration_s":0.089024,"completed_at":"2026-09-10T19:32:05.454Z","load_balancing_strategy":"primary","db_duration_s":0.000679,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.054147] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.455Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"b53bf9b589c81ecb7c9b0acc","created_at":"2026-09-10T19:32:04.586Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8022a3513994722a5a0977bd1202e262","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.594Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-b53bf9b589c81ecb7c9b0acc: start","job_status":"start","queue_duration_s":0.861697,"scheduling_latency_s":0.86172} gitlab # [ 166.131911] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.888Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"b53bf9b589c81ecb7c9b0acc","created_at":"2026-09-10T19:32:04.586Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8022a3513994722a5a0977bd1202e262","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.594Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-b53bf9b589c81ecb7c9b0acc: done: 0.432357 sec","job_status":"done","queue_duration_s":0.861697,"scheduling_latency_s":0.86172,"redis_calls":565,"redis_duration_s":0.111713,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000608,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001952,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.109153,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.359642,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000673549,"duration_s":0.432357,"completed_at":"2026-09-10T19:32:05.888Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000582,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.168356] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.890Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"baac92795ddd45ea816203ed","created_at":"2026-09-10T19:32:04.598Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dbd36f6d25e5132c60731fad99565e4c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.602Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-baac92795ddd45ea816203ed: start","job_status":"start","queue_duration_s":1.287102,"scheduling_latency_s":1.287124} gitlab # [ 166.169463] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.901Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 166.170314] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.907Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"baac92795ddd45ea816203ed","created_at":"2026-09-10T19:32:04.598Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dbd36f6d25e5132c60731fad99565e4c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.602Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-baac92795ddd45ea816203ed: done: 0.017913 sec","job_status":"done","queue_duration_s":1.287102,"scheduling_latency_s":1.287124,"redis_calls":10,"redis_duration_s":0.002291,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000477,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00137,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000444,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010122,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000410946,"duration_s":0.017913,"completed_at":"2026-09-10T19:32:05.907Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000426,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.251280] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.909Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"e7a01c1dc97415d08fac2876","created_at":"2026-09-10T19:32:04.612Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"09ad138199ce0799642de5503fc0c418","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.618Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-e7a01c1dc97415d08fac2876: start","job_status":"start","queue_duration_s":1.291337,"scheduling_latency_s":1.291357} gitlab # [ 166.252625] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.920Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 166.253975] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.926Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"e7a01c1dc97415d08fac2876","created_at":"2026-09-10T19:32:04.612Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"09ad138199ce0799642de5503fc0c418","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.618Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-e7a01c1dc97415d08fac2876: done: 0.017 sec","job_status":"done","queue_duration_s":1.291337,"scheduling_latency_s":1.291357,"redis_calls":10,"redis_duration_s":0.002287,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000587,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001245,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000455,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010013,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000389994,"duration_s":0.017,"completed_at":"2026-09-10T19:32:05.926Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000474,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.345431] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.927Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"c86dca5f98d9115524bef2f9","created_at":"2026-09-10T19:32:04.640Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ef5a0c2ec3127546fcddb50479f5a25","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.653Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-c86dca5f98d9115524bef2f9: start","job_status":"start","queue_duration_s":1.274761,"scheduling_latency_s":1.274783} gitlab # [ 166.346881] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.948Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"c86dca5f98d9115524bef2f9","created_at":"2026-09-10T19:32:04.640Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ef5a0c2ec3127546fcddb50479f5a25","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.653Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-c86dca5f98d9115524bef2f9: done: 0.021074 sec","job_status":"done","queue_duration_s":1.274761,"scheduling_latency_s":1.274783,"redis_calls":12,"redis_duration_s":0.003084,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000903,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001319,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000862,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012113,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000366806,"duration_s":0.021074,"completed_at":"2026-09-10T19:32:05.948Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001722,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.421122] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.950Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"8d123a7e97d660b851383ad9","created_at":"2026-09-10T19:32:04.672Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5473a7c7900e54dfb065a2e5621e036b","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.687Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-8d123a7e97d660b851383ad9: start","job_status":"start","queue_duration_s":1.262807,"scheduling_latency_s":1.262829} gitlab # [ 166.490836] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.976Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"8d123a7e97d660b851383ad9","created_at":"2026-09-10T19:32:04.672Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5473a7c7900e54dfb065a2e5621e036b","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.687Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-8d123a7e97d660b851383ad9: done: 0.026411 sec","job_status":"done","queue_duration_s":1.262807,"scheduling_latency_s":1.262829,"redis_calls":13,"redis_duration_s":0.002932,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000436,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001557,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000939,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016763,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.006830477000000001,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.026411,"completed_at":"2026-09-10T19:32:05.976Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002334,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 166.495323] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.978Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"153f56d61db93873fb8d2959","created_at":"2026-09-10T19:32:04.706Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1903936af27265afdc5ef21b59fcf2f0","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.720Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-153f56d61db93873fb8d2959: start","job_status":"start","queue_duration_s":1.257345,"scheduling_latency_s":1.257366} gitlab # [ 166.579544] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:05.998Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"153f56d61db93873fb8d2959","created_at":"2026-09-10T19:32:04.706Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1903936af27265afdc5ef21b59fcf2f0","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.720Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-153f56d61db93873fb8d2959: done: 0.020187 sec","job_status":"done","queue_duration_s":1.257345,"scheduling_latency_s":1.257366,"redis_calls":10,"redis_duration_s":0.002145,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000406,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001286,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000453,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012317,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000338869,"duration_s":0.020187,"completed_at":"2026-09-10T19:32:05.998Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001917,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.583265] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.000Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"47227805c6fb3ca40dd77780","created_at":"2026-09-10T19:32:04.734Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8adc28094e6b1a68ea602856d815c3e5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.746Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-47227805c6fb3ca40dd77780: start","job_status":"start","queue_duration_s":1.253873,"scheduling_latency_s":1.253894} gitlab # [ 166.658610] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.036Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"47227805c6fb3ca40dd77780","created_at":"2026-09-10T19:32:04.734Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8adc28094e6b1a68ea602856d815c3e5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.746Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-47227805c6fb3ca40dd77780: done: 0.036142 sec","job_status":"done","queue_duration_s":1.253873,"scheduling_latency_s":1.253894,"redis_calls":19,"redis_duration_s":0.009363,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002261,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000359,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.005116,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001627,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019642,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000424356,"duration_s":0.036142,"completed_at":"2026-09-10T19:32:06.036Z","load_balancing_strategy":"primary","db_duration_s":0.003875,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.670507] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.037Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"096c4b868295d88349a63886","created_at":"2026-09-10T19:32:04.758Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3c115e33bb158d12ea84895b4ff0a2dd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.768Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-096c4b868295d88349a63886: start","job_status":"start","queue_duration_s":1.269596,"scheduling_latency_s":1.269617} gitlab # [ 166.757260] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.048Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 166.758907] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.055Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"096c4b868295d88349a63886","created_at":"2026-09-10T19:32:04.758Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3c115e33bb158d12ea84895b4ff0a2dd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.768Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-096c4b868295d88349a63886: done: 0.017197 sec","job_status":"done","queue_duration_s":1.269596,"scheduling_latency_s":1.269617,"redis_calls":10,"redis_duration_s":0.002371,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000506,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001372,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000493,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009514,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000339708,"duration_s":0.017197,"completed_at":"2026-09-10T19:32:06.054Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00044,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.763875] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.056Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"9fe61ea4bfa28c7acb5ef518","created_at":"2026-09-10T19:32:04.784Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"67d0bff673d3622fbc5e9695a30c7164","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.795Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-9fe61ea4bfa28c7acb5ef518: start","job_status":"start","queue_duration_s":1.260929,"scheduling_latency_s":1.26095} gitlab # [ 166.784468] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.074Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"9fe61ea4bfa28c7acb5ef518","created_at":"2026-09-10T19:32:04.784Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"67d0bff673d3622fbc5e9695a30c7164","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.795Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-9fe61ea4bfa28c7acb5ef518: done: 0.017705 sec","job_status":"done","queue_duration_s":1.260929,"scheduling_latency_s":1.26095,"redis_calls":12,"redis_duration_s":0.002587,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000479,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000431,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001217,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00046,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010388,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000342502,"duration_s":0.017705,"completed_at":"2026-09-10T19:32:06.074Z","load_balancing_strategy":"primary","db_duration_s":0.000398,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.862298] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.075Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"ab91bfc12ab88c23e86a6be4","created_at":"2026-09-10T19:32:04.818Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"78dc447667060303b11ddbca5e2f3fa6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.830Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-ab91bfc12ab88c23e86a6be4: start","job_status":"start","queue_duration_s":1.245401,"scheduling_latency_s":1.245422} gitlab # [ 166.873269] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.107Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"ab91bfc12ab88c23e86a6be4","created_at":"2026-09-10T19:32:04.818Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"78dc447667060303b11ddbca5e2f3fa6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.830Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-ab91bfc12ab88c23e86a6be4: done: 0.031961 sec","job_status":"done","queue_duration_s":1.245401,"scheduling_latency_s":1.245422,"redis_calls":12,"redis_duration_s":0.004275,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000396,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002615,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001264,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01759,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000505092,"exclusive_lock_hold_duration_s":0.008545500000000001,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.031961,"completed_at":"2026-09-10T19:32:06.107Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006718,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.942555] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.109Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"5470ee9300e77779c690337f","created_at":"2026-09-10T19:32:04.840Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5b388a5c67a49273e82860f05a37f39d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.850Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-5470ee9300e77779c690337f: start","job_status":"start","queue_duration_s":1.259561,"scheduling_latency_s":1.259585} gitlab # [ 167.012505] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.132Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"5470ee9300e77779c690337f","created_at":"2026-09-10T19:32:04.840Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5b388a5c67a49273e82860f05a37f39d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.850Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-5470ee9300e77779c690337f: done: 0.02284 sec","job_status":"done","queue_duration_s":1.259561,"scheduling_latency_s":1.259585,"redis_calls":12,"redis_duration_s":0.002687,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000426,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001307,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000954,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013959,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0004752,"exclusive_lock_hold_duration_s":0.005518858,"duration_s":0.02284,"completed_at":"2026-09-10T19:32:06.132Z","load_balancing_strategy":"primary","db_duration_s":0.002163,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.082443] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.134Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"e61274b35f1e1df29dd1c4fe","created_at":"2026-09-10T19:32:04.858Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0cac18f43df14f53de4190001ddad199","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.864Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-e61274b35f1e1df29dd1c4fe: start","job_status":"start","queue_duration_s":1.269555,"scheduling_latency_s":1.269577} gitlab # [ 167.083279] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.157Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"e61274b35f1e1df29dd1c4fe","created_at":"2026-09-10T19:32:04.858Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0cac18f43df14f53de4190001ddad199","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.864Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-e61274b35f1e1df29dd1c4fe: done: 0.0229 sec","job_status":"done","queue_duration_s":1.269555,"scheduling_latency_s":1.269577,"redis_calls":12,"redis_duration_s":0.002549,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000391,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001272,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000886,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013668,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00045732,"exclusive_lock_hold_duration_s":0.004412013,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.0229,"completed_at":"2026-09-10T19:32:06.156Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001798,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.154326] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.158Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"be88506c3bdf2780c2d24681","created_at":"2026-09-10T19:32:04.884Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b789f4927458b4be32c08b2c9303e7eb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-10T19:32:04.899Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-be88506c3bdf2780c2d24681: start","job_status":"start","queue_duration_s":1.259483,"scheduling_latency_s":1.259504} gitlab # [ 167.155349] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.171Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 167.156450] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.179Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"be88506c3bdf2780c2d24681","created_at":"2026-09-10T19:32:04.884Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b789f4927458b4be32c08b2c9303e7eb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-10T19:32:04.899Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-be88506c3bdf2780c2d24681: done: 0.02109 sec","job_status":"done","queue_duration_s":1.259483,"scheduling_latency_s":1.259504,"redis_calls":10,"redis_duration_s":0.0020729999999999998,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00041,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001236,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000427,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012255,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034781,"duration_s":0.02109,"completed_at":"2026-09-10T19:32:06.179Z","load_balancing_strategy":"primary","db_duration_s":0.000407,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.235437] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.182Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"9456f9680598a1e545170c8c","created_at":"2026-09-10T19:32:04.931Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f698bb1dbb43cee23d650cd6958c495b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.957Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-9456f9680598a1e545170c8c: start","job_status":"start","queue_duration_s":1.224985,"scheduling_latency_s":1.225008} gitlab # [ 167.239615] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.202Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"9456f9680598a1e545170c8c","created_at":"2026-09-10T19:32:04.931Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f698bb1dbb43cee23d650cd6958c495b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.957Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-9456f9680598a1e545170c8c: done: 0.020956 sec","job_status":"done","queue_duration_s":1.224985,"scheduling_latency_s":1.225008,"redis_calls":10,"redis_duration_s":0.002169,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000404,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001324,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000441,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012177,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000397257,"duration_s":0.020956,"completed_at":"2026-09-10T19:32:06.202Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000386,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.318301] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.205Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"d23f805eecdb873b44f0cc96","created_at":"2026-09-10T19:32:04.977Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.990Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-d23f805eecdb873b44f0cc96: start","job_status":"start","queue_duration_s":1.214192,"scheduling_latency_s":1.214216} gitlab # [ 167.320129] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.311Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"d23f805eecdb873b44f0cc96","created_at":"2026-09-10T19:32:04.977Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:04.990Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-d23f805eecdb873b44f0cc96: done: 0.106107 sec","job_status":"done","queue_duration_s":1.214192,"scheduling_latency_s":1.214216,"redis_calls":71,"redis_duration_s":0.024834000000000002,"redis_read_bytes":121,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002203,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000705,"redis_queues_read_bytes":4,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.021333,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000593,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.079131,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000359822,"duration_s":0.106107,"completed_at":"2026-09-10T19:32:06.311Z","load_balancing_strategy":"primary","db_duration_s":0.001659,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.400088] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.312Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"4b12b1493ed3a09fbd4d15bc","created_at":"2026-09-10T19:32:05.003Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3dbed708907a584176b17712f7e3b25d","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.010Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-4b12b1493ed3a09fbd4d15bc: start","job_status":"start","queue_duration_s":1.302135,"scheduling_latency_s":1.302159} gitlab # [ 167.469330] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.343Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"4b12b1493ed3a09fbd4d15bc","created_at":"2026-09-10T19:32:05.003Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-ed1028b599ce415a","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3dbed708907a584176b17712f7e3b25d","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.010Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-4b12b1493ed3a09fbd4d15bc: done: 0.031041 sec","job_status":"done","queue_duration_s":1.302135,"scheduling_latency_s":1.302159,"redis_calls":10,"redis_duration_s":0.002297,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000395,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001352,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00055,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014762,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000421003,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.031041,"completed_at":"2026-09-10T19:32:06.343Z","load_balancing_strategy":"primary","db_duration_s":0.00868,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.471675] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.345Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3ba8412fa27d35b89ffd2249","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3ba8412fa27d35b89ffd2249: start","job_status":"start","queue_duration_s":0.899519,"scheduling_latency_s":0.899542} gitlab # [ 167.540388] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.374Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3ba8412fa27d35b89ffd2249","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3ba8412fa27d35b89ffd2249: done: 0.02954 sec","job_status":"done","queue_duration_s":0.899519,"scheduling_latency_s":0.899542,"redis_calls":13,"redis_duration_s":0.003031,"redis_read_bytes":12,"redis_write_bytes":1300,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000461,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":289,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00111,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":395,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00146,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":616,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018574,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000628571,"duration_s":0.02954,"completed_at":"2026-09-10T19:32:06.374Z","load_balancing_strategy":"primary","db_duration_s":0.003889,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.551053] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.376Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4c07e246e9c212c5890fc952","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4c07e246e9c212c5890fc952: start","job_status":"start","queue_duration_s":0.930768,"scheduling_latency_s":0.930791} gitlab # [ 167.630597] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.400Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4c07e246e9c212c5890fc952","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4c07e246e9c212c5890fc952: done: 0.023977 sec","job_status":"done","queue_duration_s":0.930768,"scheduling_latency_s":0.930791,"redis_calls":10,"redis_duration_s":0.0021590000000000003,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001283,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017819,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023977,"completed_at":"2026-09-10T19:32:06.400Z","load_balancing_strategy":"primary","db_duration_s":0.00179,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.632426] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.401Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6aa01805d0403a4375a3f767","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6aa01805d0403a4375a3f767: start","job_status":"start","queue_duration_s":0.956156,"scheduling_latency_s":0.956179} gitlab # [ 167.701293] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.421Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6aa01805d0403a4375a3f767","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6aa01805d0403a4375a3f767: done: 0.019474 sec","job_status":"done","queue_duration_s":0.956156,"scheduling_latency_s":0.956179,"redis_calls":10,"redis_duration_s":0.00329,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001614,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001676,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012363,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019474,"completed_at":"2026-09-10T19:32:06.421Z","load_balancing_strategy":"primary","db_duration_s":0.001553,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.771310] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.424Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a2d0cfee2d909960c78fcb4b","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a2d0cfee2d909960c78fcb4b: start","job_status":"start","queue_duration_s":0.978442,"scheduling_latency_s":0.978465} gitlab # [ 167.772571] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.450Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a2d0cfee2d909960c78fcb4b","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a2d0cfee2d909960c78fcb4b: done: 0.026315 sec","job_status":"done","queue_duration_s":0.978442,"scheduling_latency_s":0.978465,"redis_calls":10,"redis_duration_s":0.0063230000000000005,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002134,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.004189,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026315,"completed_at":"2026-09-10T19:32:06.450Z","load_balancing_strategy":"primary","db_duration_s":0.004768,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.785747] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.451Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"838ce6e087f21ac19953667e","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-838ce6e087f21ac19953667e: start","job_status":"start","queue_duration_s":1.006074,"scheduling_latency_s":1.006129} gitlab # [ 167.805309] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.468Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"838ce6e087f21ac19953667e","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-838ce6e087f21ac19953667e: done: 0.017283 sec","job_status":"done","queue_duration_s":1.006074,"scheduling_latency_s":1.006129,"redis_calls":10,"redis_duration_s":0.002095,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000844,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001251,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011602,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017283,"completed_at":"2026-09-10T19:32:06.468Z","load_balancing_strategy":"primary","db_duration_s":0.001546,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.881156] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.470Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c558cebc598f44458d0d5b30","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c558cebc598f44458d0d5b30: start","job_status":"start","queue_duration_s":1.024871,"scheduling_latency_s":1.024893} gitlab # [ 167.883532] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.487Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c558cebc598f44458d0d5b30","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c558cebc598f44458d0d5b30: done: 0.017028 sec","job_status":"done","queue_duration_s":1.024871,"scheduling_latency_s":1.024893,"redis_calls":10,"redis_duration_s":0.0021190000000000002,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000835,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001284,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011501,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017028,"completed_at":"2026-09-10T19:32:06.487Z","load_balancing_strategy":"primary","db_duration_s":0.00149,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.955222] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.489Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"26294a8e0555be0c91ae3111","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1[ 168.294802] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 1730403523 wd_nsec: 1730403460 gitlab # 676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-26294a8e0555be0c91ae3111: start","job_status":"start","queue_duration_s":1.043453,"scheduling_latency_s":1.043475} gitlab # [ 168.032995] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.505Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"26294a8e0555be0c91ae3111","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-26294a8e0555be0c91ae3111: done: 0.016826 sec","job_status":"done","queue_duration_s":1.043453,"scheduling_latency_s":1.043475,"redis_calls":10,"redis_duration_s":0.0023169999999999996,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001107,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00121,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011607,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016826,"completed_at":"2026-09-10T19:32:06.505Z","load_balancing_strategy":"primary","db_duration_s":0.001445,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.034929] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.507Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"58af67108282c45d56901e43","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-58af67108282c45d56901e43: start","job_status":"start","queue_duration_s":1.061791,"scheduling_latency_s":1.061813} gitlab # [ 168.042950] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.532Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"58af67108282c45d56901e43","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-58af67108282c45d56901e43: done: 0.025592 sec","job_status":"done","queue_duration_s":1.061791,"scheduling_latency_s":1.061813,"redis_calls":10,"redis_duration_s":0.007195,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004452,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002743,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025592,"completed_at":"2026-09-10T19:32:06.532Z","load_balancing_strategy":"primary","db_duration_s":0.005321,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.118885] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.534Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f11d323f44d2c22c16908dd8","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f11d323f44d2c22c16908dd8: start","job_status":"start","queue_duration_s":1.089046,"scheduling_latency_s":1.089068} gitlab # [ 168.120833] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.554Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f11d323f44d2c22c16908dd8","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f11d323f44d2c22c16908dd8: done: 0.019397 sec","job_status":"done","queue_duration_s":1.089046,"scheduling_latency_s":1.089068,"redis_calls":10,"redis_duration_s":0.0023769999999999998,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001001,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001376,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012073,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019397,"completed_at":"2026-09-10T19:32:06.553Z","load_balancing_strategy":"primary","db_duration_s":0.003535,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.198996] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.555Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f44fe18986b7ce322d2376d8","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f44fe18986b7ce322d2376d8: start","job_status":"start","queue_duration_s":1.109875,"scheduling_latency_s":1.109897} gitlab # [ 168.200174] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.573Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f44fe18986b7ce322d2376d8","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f44fe18986b7ce322d2376d8: done: 0.018079 sec","job_status":"done","queue_duration_s":1.109875,"scheduling_latency_s":1.109897,"redis_calls":10,"redis_duration_s":0.002107,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0008,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001307,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011913,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018079,"completed_at":"2026-09-10T19:32:06.573Z","load_balancing_strategy":"primary","db_duration_s":0.001564,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.273903] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.574Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8a4dc9abd7e74d4901cd61cd","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8a4dc9abd7e74d4901cd61cd: start","job_status":"start","queue_duration_s":1.129274,"scheduling_latency_s":1.129295} gitlab # [ 168.277323] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.593Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8a4dc9abd7e74d4901cd61cd","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8a4dc9abd7e74d4901cd61cd: done: 0.019117 sec","job_status":"done","queue_duration_s":1.129274,"scheduling_latency_s":1.129295,"redis_calls":10,"redis_duration_s":0.002134,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000894,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00124,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012972,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019117,"completed_at":"2026-09-10T19:32:06.593Z","load_balancing_strategy":"primary","db_duration_s":0.001542,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.383258] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.595Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"13cfdc3a70579735c4da7a99","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-13cfdc3a70579735c4da7a99: start","job_status":"start","queue_duration_s":1.149701,"scheduling_latency_s":1.149724} gitlab # [ 168.384850] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.613Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"13cfdc3a70579735c4da7a99","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-13cfdc3a70579735c4da7a99: done: 0.018719 sec","job_status":"done","queue_duration_s":1.149701,"scheduling_latency_s":1.149724,"redis_calls":10,"redis_duration_s":0.00251,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001274,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001236,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01215,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018719,"completed_at":"2026-09-10T19:32:06.613Z","load_balancing_strategy":"primary","db_duration_s":0.002174,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.463792] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.615Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d3b13f2db2d786b21a5fc0a0","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d3b13f2db2d786b21a5fc0a0: start","job_status":"start","queue_duration_s":1.170065,"scheduling_latency_s":1.17013} gitlab # [ 168.464988] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.639Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d3b13f2db2d786b21a5fc0a0","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d3b13f2db2d786b21a5fc0a0: done: 0.023854 sec","job_status":"done","queue_duration_s":1.170065,"scheduling_latency_s":1.17013,"redis_calls":10,"redis_duration_s":0.005988,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00237,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003618,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012446,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023854,"completed_at":"2026-09-10T19:32:06.639Z","load_balancing_strategy":"primary","db_duration_s":0.001559,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.537115] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.641Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b7554f84d2a778e57299ac1d","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b7554f84d2a778e57299ac1d: start","job_status":"start","queue_duration_s":1.195639,"scheduling_latency_s":1.195661} gitlab # [ 168.538467] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.659Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b7554f84d2a778e57299ac1d","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b7554f84d2a778e57299ac1d: done: 0.01791 sec","job_status":"done","queue_duration_s":1.195639,"scheduling_latency_s":1.195661,"redis_calls":10,"redis_duration_s":0.002144,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000789,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001355,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012341,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01791,"completed_at":"2026-09-10T19:32:06.659Z","load_balancing_strategy":"primary","db_duration_s":0.001476,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.623801] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.660Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"1be84fc5d25d366caf9fb5d9","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-1be84fc5d25d366caf9fb5d9: start","job_status":"start","queue_duration_s":1.21518,"scheduling_latency_s":1.215201} gitlab # [ 168.624929] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.678Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"1be84fc5d25d366caf9fb5d9","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-1be84fc5d25d366caf9fb5d9: done: 0.01742 sec","job_status":"done","queue_duration_s":1.21518,"scheduling_latency_s":1.215201,"redis_calls":10,"redis_duration_s":0.0020759999999999997,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000838,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001238,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012028,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01742,"completed_at":"2026-09-10T19:32:06.678Z","load_balancing_strategy":"primary","db_duration_s":0.001568,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.711504] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.679Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5649ab24d4655781a12986b5","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5649ab24d4655781a12986b5: start","job_status":"start","queue_duration_s":1.233892,"scheduling_latency_s":1.233915} gitlab # [ 168.713045] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.696Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5649ab24d4655781a12986b5","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5649ab24d4655781a12986b5: done: 0.017388 sec","job_status":"done","queue_duration_s":1.233892,"scheduling_latency_s":1.233915,"redis_calls":10,"redis_duration_s":0.002587,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001061,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001526,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011586,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017388,"completed_at":"2026-09-10T19:32:06.696Z","load_balancing_strategy":"primary","db_duration_s":0.001461,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.782601] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.699Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"438c70e2aada7a5af5191136","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-438c70e2aada7a5af5191136: start","job_status":"start","queue_duration_s":1.254227,"scheduling_latency_s":1.25425} gitlab # [ 168.784325] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.717Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"438c70e2aada7a5af5191136","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-438c70e2aada7a5af5191136: done: 0.017286 sec","job_status":"done","queue_duration_s":1.254227,"scheduling_latency_s":1.25425,"redis_calls":10,"redis_duration_s":0.002013,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000796,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001217,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011363,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017286,"completed_at":"2026-09-10T19:32:06.717Z","load_balancing_strategy":"primary","db_duration_s":0.001444,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.854308] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.718Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6bdfc232916bb2d9bc41074f","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6bdfc232916bb2d9bc41074f: start","job_status":"start","queue_duration_s":1.272781,"scheduling_latency_s":1.272803} gitlab # [ 168.855887] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.735Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6bdfc232916bb2d9bc41074f","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6bdfc232916bb2d9bc41074f: done: 0.016816 sec","job_status":"done","queue_duration_s":1.272781,"scheduling_latency_s":1.272803,"redis_calls":10,"redis_duration_s":0.0023179999999999997,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000802,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001516,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011241,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016816,"completed_at":"2026-09-10T19:32:06.735Z","load_balancing_strategy":"primary","db_duration_s":0.001542,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.927084] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.736Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c5c638c0ba767be4730ab0b8","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c5c638c0ba767be4730ab0b8: start","job_status":"start","queue_duration_s":1.290914,"scheduling_latency_s":1.290936} gitlab # [ 168.928166] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.753Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c5c638c0ba767be4730ab0b8","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c5c638c0ba767be4730ab0b8: done: 0.016586 sec","job_status":"done","queue_duration_s":1.290914,"scheduling_latency_s":1.290936,"redis_calls":10,"redis_duration_s":0.002118,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000793,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001325,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011231,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016586,"completed_at":"2026-09-10T19:32:06.753Z","load_balancing_strategy":"primary","db_duration_s":0.001506,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.963749] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.754Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fb0cf158b1609990bba6fc4e","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fb0cf158b1609990bba6fc4e: start","job_status":"start","queue_duration_s":1.308781,"scheduling_latency_s":1.308803} gitlab # [ 169.046626] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.771Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fb0cf158b1609990bba6fc4e","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fb0cf158b1609990bba6fc4e: done: 0.017265 sec","job_status":"done","queue_duration_s":1.308781,"scheduling_latency_s":1.308803,"redis_calls":10,"redis_duration_s":0.003051,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001393,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001658,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011106,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017265,"completed_at":"2026-09-10T19:32:06.771Z","load_balancing_strategy":"primary","db_duration_s":0.001401,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 169.048876] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.773Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"78ba2c742a41d3aa2cad1e7d","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-78ba2c742a41d3aa2cad1e7d: start","job_status":"start","queue_duration_s":1.327403,"scheduling_latency_s":1.327427} gitlab # [ 169.061046] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.789Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"78ba2c742a41d3aa2cad1e7d","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-78ba2c742a41d3aa2cad1e7d: done: 0.01642 sec","job_status":"done","queue_duration_s":1.327403,"scheduling_latency_s":1.327427,"redis_calls":10,"redis_duration_s":0.0020959999999999998,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000882,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001214,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011122,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01642,"completed_at":"2026-09-10T19:32:06.789Z","load_balancing_strategy":"primary","db_duration_s":0.00145,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 169.152157] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.790Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"799b124cd87e133e79648660","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-799b124cd87e133e79648660: start","job_status":"start","queue_duration_s":1.345144,"scheduling_latency_s":1.345165} gitlab # [ 169.153460] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.806Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"799b124cd87e133e79648660","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-799b124cd87e133e79648660: done: 0.016323 sec","job_status":"done","queue_duration_s":1.345144,"scheduling_latency_s":1.345165,"redis_calls":10,"redis_duration_s":0.00211,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000788,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001322,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016323,"completed_at":"2026-09-10T19:32:06.806Z","load_balancing_strategy":"primary","db_duration_s":0.00152,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 169.227047] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.808Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7d4615c96bbca68bdd5bcbff","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7d4615c96bbca68bdd5bcbff: start","job_status":"start","queue_duration_s":1.362678,"scheduling_latency_s":1.362698} gitlab # [ 169.228423] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.824Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7d4615c96bbca68bdd5bcbff","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7d4615c96bbca68bdd5bcbff: done: 0.016208 sec","job_status":"done","queue_duration_s":1.362678,"scheduling_latency_s":1.362698,"redis_calls":10,"redis_duration_s":0.0021260000000000003,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000892,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001234,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011005,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016208,"completed_at":"2026-09-10T19:32:06.824Z","load_balancing_strategy":"primary","db_duration_s":0.001405,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 169.296193] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.826Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"37671108149f7debbbe1b66c","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-37671108149f7debbbe1b66c: start","job_status":"start","queue_duration_s":1.380395,"scheduling_latency_s":1.380415} gitlab # [ 169.297850] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.845Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"37671108149f7debbbe1b66c","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-37671108149f7debbbe1b66c: done: 0.019573 sec","job_status":"done","queue_duration_s":1.380395,"scheduling_latency_s":1.380415,"redis_calls":10,"redis_duration_s":0.005488,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003788,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.0017,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010993,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019573,"completed_at":"2026-09-10T19:32:06.845Z","load_balancing_strategy":"primary","db_duration_s":0.001443,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 169.366115] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.846Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a51c28afd98738f217310e4f","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a51c28afd98738f217310e4f: start","job_status":"start","queue_duration_s":1.4013,"scheduling_latency_s":1.401323} gitlab # [ 169.370436] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.863Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a51c28afd98738f217310e4f","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a51c28afd98738f217310e4f: done: 0.016168 sec","job_status":"done","queue_duration_s":1.4013,"scheduling_latency_s":1.401323,"redis_calls":10,"redis_duration_s":0.002081,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000786,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001295,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010925,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016168,"completed_at":"2026-09-10T19:32:06.862Z","load_balancing_strategy":"primary","db_duration_s":0.001539,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 169.430441] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.864Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e33effd6cff1d8fef87c4cd7","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e33effd6cff1d8fef87c4cd7: start","job_status":"start","queue_duration_s":1.418725,"scheduling_latency_s":1.418746} gitlab # [ 169.506040] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.880Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e33effd6cff1d8fef87c4cd7","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e33effd6cff1d8fef87c4cd7: done: 0.016033 sec","job_status":"done","queue_duration_s":1.418725,"scheduling_latency_s":1.418746,"redis_calls":10,"redis_duration_s":0.002069,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000879,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00119,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010837,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016033,"completed_at":"2026-09-10T19:32:06.880Z","load_balancing_strategy":"primary","db_duration_s":0.001419,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 169.508296] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.881Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a744722b4e25a8e5a35365ca","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a744722b4e25a8e5a35365ca: start","job_status":"start","queue_duration_s":1.436017,"scheduling_latency_s":1.436038} gitlab # [ 169.527341] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.897Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a744722b4e25a8e5a35365ca","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a744722b4e25a8e5a35365ca: done: 0.016088 sec","job_status":"done","queue_duration_s":1.436017,"scheduling_latency_s":1.436038,"redis_calls":10,"redis_duration_s":0.002075,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000834,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001241,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010871,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016088,"completed_at":"2026-09-10T19:32:06.897Z","load_balancing_strategy":"primary","db_duration_s":0.001401,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 169.645338] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.899Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d4b440ef8905b965192d3b62","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d4b440ef8905b965192d3b62: start","job_status":"start","queue_duration_s":1.453423,"scheduling_latency_s":1.453445} gitlab # [ 169.647036] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.920Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d4b440ef8905b965192d3b62","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d4b440ef8905b965192d3b62: done: 0.021695 sec","job_status":"done","queue_duration_s":1.453423,"scheduling_latency_s":1.453445,"redis_calls":10,"redis_duration_s":0.005175,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00349,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001685,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01109,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021695,"completed_at":"2026-09-10T19:32:06.920Z","load_balancing_strategy":"primary","db_duration_s":0.0038,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 169.664984] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.923Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b32b55f557cbc13f1b163566","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b32b55f557cbc13f1b163566: start","job_status":"start","queue_duration_s":1.477718,"scheduling_latency_s":1.47774} gitlab # [ 169.684205] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.939Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b32b55f557cbc13f1b163566","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b32b55f557cbc13f1b163566: done: 0.016668 sec","job_status":"done","queue_duration_s":1.477718,"scheduling_latency_s":1.47774,"redis_calls":10,"redis_duration_s":0.0021000000000000003,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000845,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001255,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011235,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016668,"completed_at":"2026-09-10T19:32:06.939Z","load_balancing_strategy":"primary","db_duration_s":0.001478,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 169.757194] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.941Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0d98993f6ba5ea2da78d39eb","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0d98993f6ba5ea2da78d39eb: start","job_status":"start","queue_duration_s":1.4957,"scheduling_latency_s":1.495724} gitlab # [ 169.761761] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.958Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0d98993f6ba5ea2da78d39eb","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0d98993f6ba5ea2da78d39eb: done: 0.017548 sec","job_status":"done","queue_duration_s":1.4957,"scheduling_latency_s":1.495724,"redis_calls":10,"redis_duration_s":0.0021999999999999997,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000856,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001344,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011955,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017548,"completed_at":"2026-09-10T19:32:06.958Z","load_balancing_strategy":"primary","db_duration_s":0.001544,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 169.838138] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.960Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8f7377cee9379ce136e0c7b3","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8f7377cee9379ce136e0c7b3: start","job_status":"start","queue_duration_s":1.514627,"scheduling_latency_s":1.514651} gitlab # [ 169.839438] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.977Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8f7377cee9379ce136e0c7b3","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8f7377cee9379ce136e0c7b3: done: 0.017739 sec","job_status":"done","queue_duration_s":1.514627,"scheduling_latency_s":1.514651,"redis_calls":10,"redis_duration_s":0.002248,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000993,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001255,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011947,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017739,"completed_at":"2026-09-10T19:32:06.977Z","load_balancing_strategy":"primary","db_duration_s":0.001424,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 169.926807] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:06.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"1f88a0c63800412138b37940","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-1f88a0c63800412138b37940: start","job_status":"start","queue_duration_s":1.533709,"scheduling_latency_s":1.533732} gitlab # [ 169.927961] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.001Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"1f88a0c63800412138b37940","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-1f88a0c63800412138b37940: done: 0.022495 sec","job_status":"done","queue_duration_s":1.533709,"scheduling_latency_s":1.533732,"redis_calls":10,"redis_duration_s":0.0051400000000000005,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00126,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00388,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011632,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022495,"completed_at":"2026-09-10T19:32:07.001Z","load_balancing_strategy":"primary","db_duration_s":0.003698,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 170.002413] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.007Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"36f2fc4699fab7e6f4fb3a30","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-36f2fc4699fab7e6f4fb3a30: start","job_status":"start","queue_duration_s":1.561506,"scheduling_latency_s":1.56153} gitlab # [ 170.004039] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.024Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"36f2fc4699fab7e6f4fb3a30","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-36f2fc4699fab7e6f4fb3a30: done: 0.017214 sec","job_status":"done","queue_duration_s":1.561506,"scheduling_latency_s":1.56153,"redis_calls":10,"redis_duration_s":0.002107,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000812,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001295,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011559,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017214,"completed_at":"2026-09-10T19:32:07.024Z","load_balancing_strategy":"primary","db_duration_s":0.001571,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 170.111162] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.025Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0a2a807fd9a7038f663e9abb","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0a2a807fd9a7038f663e9abb: start","job_status":"start","queue_duration_s":1.580027,"scheduling_latency_s":1.580049} gitlab # [ 170.112444] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.042Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0a2a807fd9a7038f663e9abb","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0a2a807fd9a7038f663e9abb: done: 0.01673 sec","job_status":"done","queue_duration_s":1.580027,"scheduling_latency_s":1.580049,"redis_calls":10,"redis_duration_s":0.002103,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000797,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001306,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01673,"completed_at":"2026-09-10T19:32:07.042Z","load_balancing_strategy":"primary","db_duration_s":0.001483,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 170.131137] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.043Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b139e99165b1136e6306d264","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b139e99165b1136e6306d264: start","job_status":"start","queue_duration_s":1.598036,"scheduling_latency_s":1.598058} gitlab # [ 170.205349] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.060Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b139e99165b1136e6306d264","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b139e99165b1136e6306d264: done: 0.016744 sec","job_status":"done","queue_duration_s":1.598036,"scheduling_latency_s":1.598058,"redis_calls":10,"redis_duration_s":0.002115,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000791,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001324,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011366,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016744,"completed_at":"2026-09-10T19:32:07.060Z","load_balancing_strategy":"primary","db_duration_s":0.001445,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 170.208178] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.061Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"73db890e451f95757df68c06","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-73db890e451f95757df68c06: start","job_status":"start","queue_duration_s":1.616302,"scheduling_latency_s":1.616325} gitlab # [ 170.227295] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.085Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"73db890e451f95757df68c06","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-73db890e451f95757df68c06: done: 0.024094 sec","job_status":"done","queue_duration_s":1.616302,"scheduling_latency_s":1.616325,"redis_calls":10,"redis_duration_s":0.006592,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001593,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.004999,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011944,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024094,"completed_at":"2026-09-10T19:32:07.085Z","load_balancing_strategy":"primary","db_duration_s":0.003902,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 170.345328] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.089Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ff773a15443c394aab77d9e9","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ff773a15443c394aab77d9e9: start","job_status":"start","queue_duration_s":1.643453,"scheduling_latency_s":1.643477} gitlab # [ 170.346897] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.106Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ff773a15443c394aab77d9e9","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ff773a15443c394aab77d9e9: done: 0.017058 sec","job_status":"done","queue_duration_s":1.643453,"scheduling_latency_s":1.643477,"redis_calls":10,"redis_duration_s":0.00209,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00084,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00125,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011604,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017058,"completed_at":"2026-09-10T19:32:07.106Z","load_balancing_strategy":"primary","db_duration_s":0.001495,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 170.364272] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.107Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d34bb7bebea2f52a9a08c79d","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d34bb7bebea2f52a9a08c79d: start","job_status":"start","queue_duration_s":1.661797,"scheduling_latency_s":1.661819} gitlab # [ 170.383398] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.124Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d34bb7bebea2f52a9a08c79d","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d34bb7bebea2f52a9a08c79d: done: 0.016712 sec","job_status":"done","queue_duration_s":1.661797,"scheduling_latency_s":1.661819,"redis_calls":10,"redis_duration_s":0.002067,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000795,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001272,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011361,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016712,"completed_at":"2026-09-10T19:32:07.124Z","load_balancing_strategy":"primary","db_duration_s":0.001577,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 170.466046] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.125Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7695c3762ec8f5b348af6f96","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7695c3762ec8f5b348af6f96: start","job_status":"start","queue_duration_s":1.679771,"scheduling_latency_s":1.679791} gitlab # [ 170.467483] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.142Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7695c3762ec8f5b348af6f96","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7695c3762ec8f5b348af6f96: done: 0.016863 sec","job_status":"done","queue_duration_s":1.679771,"scheduling_latency_s":1.679791,"redis_calls":10,"redis_duration_s":0.002328,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00079,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001538,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011093,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016863,"completed_at":"2026-09-10T19:32:07.142Z","load_balancing_strategy":"primary","db_duration_s":0.001485,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 170.574457] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.143Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"03dcfe8550f525e10f1871b5","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-03dcfe8550f525e10f1871b5: start","job_status":"start","queue_duration_s":1.697947,"scheduling_latency_s":1.697968} gitlab # [ 170.576092] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.163Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"03dcfe8550f525e10f1871b5","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-03dcfe8550f525e10f1871b5: done: 0.020172 sec","job_status":"done","queue_duration_s":1.697947,"scheduling_latency_s":1.697968,"redis_calls":10,"redis_duration_s":0.002156,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000871,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001285,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011181,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020172,"completed_at":"2026-09-10T19:32:07.163Z","load_balancing_strategy":"primary","db_duration_s":0.004899,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 170.649148] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.165Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fa66659126b6f5069d5c2ead","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fa66659126b6f5069d5c2ead: start","job_status":"start","queue_duration_s":1.719435,"scheduling_latency_s":1.719456} gitlab # [ 170.650244] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.181Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fa66659126b6f5069d5c2ead","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fa66659126b6f5069d5c2ead: done: 0.016449 sec","job_status":"done","queue_duration_s":1.719435,"scheduling_latency_s":1.719456,"redis_calls":10,"redis_duration_s":0.002093,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000888,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001205,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016449,"completed_at":"2026-09-10T19:32:07.181Z","load_balancing_strategy":"primary","db_duration_s":0.001427,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 170.718144] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.182Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3c4dbe9c4470331893a918c9","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3c4dbe9c4470331893a918c9: start","job_status":"start","queue_duration_s":1.73734,"scheduling_latency_s":1.737363} gitlab # [ 170.719513] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.200Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3c4dbe9c4470331893a918c9","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3c4dbe9c4470331893a918c9: done: 0.01786 sec","job_status":"done","queue_duration_s":1.73734,"scheduling_latency_s":1.737363,"redis_calls":10,"redis_duration_s":0.00209,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000836,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001254,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011792,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01786,"completed_at":"2026-09-10T19:32:07.200Z","load_balancing_strategy":"primary","db_duration_s":0.002166,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 170.821624] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.202Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e7bd808c847b909fd387cbdf","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e7bd808c847b909fd387cbdf: start","job_status":"start","queue_duration_s":1.756618,"scheduling_latency_s":1.756639} gitlab # [ 170.823154] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.218Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e7bd808c847b909fd387cbdf","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e7bd808c847b909fd387cbdf: done: 0.016439 sec","job_status":"done","queue_duration_s":1.756618,"scheduling_latency_s":1.756639,"redis_calls":10,"redis_duration_s":0.002065,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00083,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001235,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011317,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016439,"completed_at":"2026-09-10T19:32:07.218Z","load_balancing_strategy":"primary","db_duration_s":0.001426,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 170.836448] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.220Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"22e1b75e4d07f1cb790c7f2e","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-22e1b75e4d07f1cb790c7f2e: start","job_status":"start","queue_duration_s":1.7745,"scheduling_latency_s":1.774522} gitlab # [ 170.912224] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.237Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"22e1b75e4d07f1cb790c7f2e","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-22e1b75e4d07f1cb790c7f2e: done: 0.017859 sec","job_status":"done","queue_duration_s":1.7745,"scheduling_latency_s":1.774522,"redis_calls":10,"redis_duration_s":0.00338,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000938,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002442,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011383,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017859,"completed_at":"2026-09-10T19:32:07.237Z","load_balancing_strategy":"primary","db_duration_s":0.001457,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 170.915136] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.239Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a4d3153b55580b4e75e31ec0","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a4d3153b55580b4e75e31ec0: start","job_status":"start","queue_duration_s":1.793662,"scheduling_latency_s":1.793686} gitlab # [ 170.934287] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.256Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a4d3153b55580b4e75e31ec0","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a4d3153b55580b4e75e31ec0: done: 0.017654 sec","job_status":"done","queue_duration_s":1.793662,"scheduling_latency_s":1.793686,"redis_calls":10,"redis_duration_s":0.002188,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00085,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001338,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011979,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017654,"completed_at":"2026-09-10T19:32:07.256Z","load_balancing_strategy":"primary","db_duration_s":0.001596,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.004111] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.258Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9c768e10d60617abecba2111","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9c768e10d60617abecba2111: start","job_status":"start","queue_duration_s":1.812647,"scheduling_latency_s":1.81267} gitlab # [ 171.012577] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.276Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9c768e10d60617abecba2111","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9c768e10d60617abecba2111: done: 0.017965 sec","job_status":"done","queue_duration_s":1.812647,"scheduling_latency_s":1.81267,"redis_calls":10,"redis_duration_s":0.002312,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001312,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012034,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017965,"completed_at":"2026-09-10T19:32:07.276Z","load_balancing_strategy":"primary","db_duration_s":0.001437,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.085550] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.277Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fee099960ee41924670934b9","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fee099960ee41924670934b9: start","job_status":"start","queue_duration_s":1.831971,"scheduling_latency_s":1.831995} gitlab # [ 171.091589] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.295Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fee099960ee41924670934b9","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fee099960ee41924670934b9: done: 0.017765 sec","job_status":"done","queue_duration_s":1.831971,"scheduling_latency_s":1.831995,"redis_calls":10,"redis_duration_s":0.00211,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000858,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001252,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011838,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017765,"completed_at":"2026-09-10T19:32:07.295Z","load_balancing_strategy":"primary","db_duration_s":0.001533,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.198276] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.301Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"98fe0d305d7c11e89731088e","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-98fe0d305d7c11e89731088e: start","job_status":"start","queue_duration_s":1.855668,"scheduling_latency_s":1.855692} gitlab # [ 171.199830] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.325Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"98fe0d305d7c11e89731088e","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-98fe0d305d7c11e89731088e: done: 0.024032 sec","job_status":"done","queue_duration_s":1.855668,"scheduling_latency_s":1.855692,"redis_calls":10,"redis_duration_s":0.005935,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001208,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.004727,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011997,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024032,"completed_at":"2026-09-10T19:32:07.325Z","load_balancing_strategy":"primary","db_duration_s":0.004951,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.275381] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.326Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b010b49c6e2ca527d8875fbf","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b010b49c6e2ca527d8875fbf: start","job_status":"start","queue_duration_s":1.881192,"scheduling_latency_s":1.881223} gitlab # [ 171.277081] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.344Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b010b49c6e2ca527d8875fbf","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b010b49c6e2ca527d8875fbf: done: 0.017808 sec","job_status":"done","queue_duration_s":1.881192,"scheduling_latency_s":1.881223,"redis_calls":10,"redis_duration_s":0.002128,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000901,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001227,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012078,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017808,"completed_at":"2026-09-10T19:32:07.344Z","load_balancing_strategy":"primary","db_duration_s":0.001531,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.344802] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.346Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c1813bb879e28f3154c244dc","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c1813bb879e28f3154c244dc: start","job_status":"start","queue_duration_s":1.900522,"scheduling_latency_s":1.900545} gitlab # [ 171.345931] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.364Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c1813bb879e28f3154c244dc","created_at":"2026-09-10T19:32:05.376Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-c54fab6cf9194d65","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"671f236a3f8975a53448e19a88abdde4","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:05.445Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c1813bb879e28f3154c244dc: done: 0.017974 sec","job_status":"done","queue_duration_s":1.900522,"scheduling_latency_s":1.900545,"redis_calls":10,"redis_duration_s":0.002014,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000808,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001206,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012418,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017974,"completed_at":"2026-09-10T19:32:07.364Z","load_balancing_strategy":"primary","db_duration_s":0.001433,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.413352] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.365Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"bd689d800fd0854efea318c4","created_at":"2026-09-10T19:32:06.020Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-0e1fdd5df16548b3","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8adc28094e6b1a68ea602856d815c3e5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.030Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-bd689d800fd0854efea318c4: start","job_status":"start","queue_duration_s":1.335142,"scheduling_latency_s":1.335165} gitlab # [ 171.414862] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.380Z","message":"No existing merge request to be cleaned up."} gitlab # [ 171.416079] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.394Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"bd689d800fd0854efea318c4","created_at":"2026-09-10T19:32:06.020Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-0e1fdd5df16548b3","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8adc28094e6b1a68ea602856d815c3e5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.030Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-bd689d800fd0854efea318c4: done: 0.029258 sec","job_status":"done","queue_duration_s":1.335142,"scheduling_latency_s":1.335165,"redis_calls":13,"redis_duration_s":0.0063219999999999995,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000465,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004063,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001794,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016608,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00080988,"duration_s":0.029258,"completed_at":"2026-09-10T19:32:07.394Z","load_balancing_strategy":"primary","db_duration_s":0.002949,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.508154] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.396Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d69df610c0c18f2d195ac028","created_at":"2026-09-10T19:32:06.020Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-0e1fdd5df16548b3","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8adc28094e6b1a68ea602856d815c3e5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.030Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d69df610c0c18f2d195ac028: start","job_status":"start","queue_duration_s":1.366006,"scheduling_latency_s":1.36603} gitlab # [ 171.509406] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.407Z","message":"No existing merge request to be cleaned up."} gitlab # [ 171.510323] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.414Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d69df610c0c18f2d195ac028","created_at":"2026-09-10T19:32:06.020Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-0e1fdd5df16548b3","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8adc28094e6b1a68ea602856d815c3e5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.030Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d69df610c0c18f2d195ac028: done: 0.018348 sec","job_status":"done","queue_duration_s":1.366006,"scheduling_latency_s":1.36603,"redis_calls":10,"redis_duration_s":0.002221,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000859,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001362,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012177,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018348,"completed_at":"2026-09-10T19:32:07.414Z","load_balancing_strategy":"primary","db_duration_s":0.001591,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.604384] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.416Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"da5203e571e31f2c871abb0a","created_at":"2026-09-10T19:32:06.020Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-0e1fdd5df16548b3","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8adc28094e6b1a68ea602856d815c3e5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.030Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-da5203e571e31f2c871abb0a: start","job_status":"start","queue_duration_s":1.385711,"scheduling_latency_s":1.385734} gitlab # [ 171.605871] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.426Z","message":"No existing merge request to be cleaned up."} gitlab # [ 171.607133] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.433Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"da5203e571e31f2c871abb0a","created_at":"2026-09-10T19:32:06.020Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-0e1fdd5df16548b3","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8adc28094e6b1a68ea602856d815c3e5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.030Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-da5203e571e31f2c871abb0a: done: 0.017749 sec","job_status":"done","queue_duration_s":1.385711,"scheduling_latency_s":1.385734,"redis_calls":10,"redis_duration_s":0.002153,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000859,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001294,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011715,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017749,"completed_at":"2026-09-10T19:32:07.433Z","load_balancing_strategy":"primary","db_duration_s":0.001523,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.694265] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.435Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ce7170228eb510f1e931030e","created_at":"2026-09-10T19:32:06.020Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-0e1fdd5df16548b3","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8adc28094e6b1a68ea602856d815c3e5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.030Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ce7170228eb510f1e931030e: start","job_status":"start","queue_duration_s":1.404802,"scheduling_latency_s":1.404826} gitlab # [ 171.695331] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.445Z","message":"No existing merge request to be cleaned up."} gitlab # [ 171.695916] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.452Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ce7170228eb510f1e931030e","created_at":"2026-09-10T19:32:06.020Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-0e1fdd5df16548b3","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8adc28094e6b1a68ea602856d815c3e5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.030Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ce7170228eb510f1e931030e: done: 0.017737 sec","job_status":"done","queue_duration_s":1.404802,"scheduling_latency_s":1.404826,"redis_calls":10,"redis_duration_s":0.002132,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000863,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001269,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011787,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017737,"completed_at":"2026-09-10T19:32:07.452Z","load_balancing_strategy":"primary","db_duration_s":0.001692,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.777257] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.454Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dedb70dec20076d9e335096f","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dedb70dec20076d9e335096f: start","job_status":"start","queue_duration_s":1.151937,"scheduling_latency_s":1.151962} gitlab # [ 171.778843] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.478Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dedb70dec20076d9e335096f","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dedb70dec20076d9e335096f: done: 0.024069 sec","job_status":"done","queue_duration_s":1.151937,"scheduling_latency_s":1.151962,"redis_calls":16,"redis_duration_s":0.004586,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000413,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001474,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002699,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013945,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000357866,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024069,"completed_at":"2026-09-10T19:32:07.478Z","load_balancing_strategy":"primary","db_duration_s":0.000389,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.850597] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.481Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bebdf5ffd82c935ed67257c8","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bebdf5ffd82c935ed67257c8: start","job_status":"start","queue_duration_s":1.17889,"scheduling_latency_s":1.178914} gitlab # [ 171.851981] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.502Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bebdf5ffd82c935ed67257c8","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bebdf5ffd82c935ed67257c8: done: 0.020784 sec","job_status":"done","queue_duration_s":1.17889,"scheduling_latency_s":1.178914,"redis_calls":13,"redis_duration_s":0.003522,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00091,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002612,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011233,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020784,"completed_at":"2026-09-10T19:32:07.501Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.925331] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.503Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c9525f35bb16c3a641bf3fb","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c9525f35bb16c3a641bf3fb: start","job_status":"start","queue_duration_s":1.201197,"scheduling_latency_s":1.201221} gitlab # [ 171.926917] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.519Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c9525f35bb16c3a641bf3fb","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c9525f35bb16c3a641bf3fb: done: 0.016345 sec","job_status":"done","queue_duration_s":1.201197,"scheduling_latency_s":1.201221,"redis_calls":13,"redis_duration_s":0.0026609999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00085,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001811,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016345,"completed_at":"2026-09-10T19:32:07.519Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 171.960242] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.521Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34e4b48740628285e5564dbc","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34e4b48740628285e5564dbc: start","job_status":"start","queue_duration_s":1.218869,"scheduling_latency_s":1.218893} gitlab # [ 172.044241] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.544Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34e4b48740628285e5564dbc","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34e4b48740628285e5564dbc: done: 0.023678 sec","job_status":"done","queue_duration_s":1.218869,"scheduling_latency_s":1.218893,"redis_calls":13,"redis_duration_s":0.009731,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002964,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006767,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011228,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023678,"completed_at":"2026-09-10T19:32:07.544Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.045979] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.548Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b0572fcd546fb0becd59580","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b0572fcd546fb0becd59580: start","job_status":"start","queue_duration_s":1.245823,"scheduling_latency_s":1.245848} gitlab # [ 172.121869] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.572Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b0572fcd546fb0becd59580","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b0572fcd546fb0becd59580: done: 0.024685 sec","job_status":"done","queue_duration_s":1.245823,"scheduling_latency_s":1.245848,"redis_calls":13,"redis_duration_s":0.009558,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0026,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006958,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011241,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024685,"completed_at":"2026-09-10T19:32:07.572Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.123932] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.574Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcb07f0fb11ae0aaa35f246a","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcb07f0fb11ae0aaa35f246a: start","job_status":"start","queue_duration_s":1.271887,"scheduling_latency_s":1.271912} gitlab # [ 172.137349] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.591Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcb07f0fb11ae0aaa35f246a","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcb07f0fb11ae0aaa35f246a: done: 0.017706 sec","job_status":"done","queue_duration_s":1.271887,"scheduling_latency_s":1.271912,"redis_calls":13,"redis_duration_s":0.002973,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000981,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001992,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011956,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017706,"completed_at":"2026-09-10T19:32:07.591Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.255332] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.593Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eca73f5f42d21c36a4e8386c","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eca73f5f42d21c36a4e8386c: start","job_status":"start","queue_duration_s":1.290902,"scheduling_latency_s":1.290925} gitlab # [ 172.256958] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.609Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eca73f5f42d21c36a4e8386c","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eca73f5f42d21c36a4e8386c: done: 0.01604 sec","job_status":"done","queue_duration_s":1.290902,"scheduling_latency_s":1.290925,"redis_calls":13,"redis_duration_s":0.002849,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000828,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010426,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01604,"completed_at":"2026-09-10T19:32:07.609Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.323577] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.610Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9f2e41ad17ff61aa1cdf13d","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9f2e41ad17ff61aa1cdf13d: start","job_status":"start","queue_duration_s":1.308217,"scheduling_latency_s":1.30824} gitlab # [ 172.324889] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.626Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9f2e41ad17ff61aa1cdf13d","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9f2e41ad17ff61aa1cdf13d: done: 0.015747 sec","job_status":"done","queue_duration_s":1.308217,"scheduling_latency_s":1.30824,"redis_calls":13,"redis_duration_s":0.002709,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000836,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001873,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010452,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015747,"completed_at":"2026-09-10T19:32:07.626Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.356739] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.627Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"924d35b0f5df0a4d4df62c0e","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-924d35b0f5df0a4d4df62c0e: start","job_status":"start","queue_duration_s":1.325253,"scheduling_latency_s":1.325275} gitlab # [ 172.430458] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.649Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"924d35b0f5df0a4d4df62c0e","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-924d35b0f5df0a4d4df62c0e: done: 0.022201 sec","job_status":"done","queue_duration_s":1.325253,"scheduling_latency_s":1.325275,"redis_calls":13,"redis_duration_s":0.008726999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000898,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.007829,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010872,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022201,"completed_at":"2026-09-10T19:32:07.649Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.499443] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.651Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c19d5efc9a2313e7c9d54508","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c19d5efc9a2313e7c9d54508: start","job_status":"start","queue_duration_s":1.349007,"scheduling_latency_s":1.349031} gitlab # [ 172.500544] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.667Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c19d5efc9a2313e7c9d54508","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c19d5efc9a2313e7c9d54508: done: 0.016167 sec","job_status":"done","queue_duration_s":1.349007,"scheduling_latency_s":1.349031,"redis_calls":13,"redis_duration_s":0.0026420000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0008,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001842,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010715,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016167,"completed_at":"2026-09-10T19:32:07.667Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.515768] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.669Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0709aad7f7a80156ecd681a","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0709aad7f7a80156ecd681a: start","job_status":"start","queue_duration_s":1.366715,"scheduling_latency_s":1.366736} gitlab # [ 172.535384] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.684Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0709aad7f7a80156ecd681a","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0709aad7f7a80156ecd681a: done: 0.015895 sec","job_status":"done","queue_duration_s":1.366715,"scheduling_latency_s":1.366736,"redis_calls":13,"redis_duration_s":0.002619,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000837,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001782,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010758,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015895,"completed_at":"2026-09-10T19:32:07.684Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.605054] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.686Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21d04dfc506a98cd3fa21096","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21d04dfc506a98cd3fa21096: start","job_status":"start","queue_duration_s":1.384083,"scheduling_latency_s":1.384106} gitlab # [ 172.614156] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.702Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21d04dfc506a98cd3fa21096","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21d04dfc506a98cd3fa21096: done: 0.015685 sec","job_status":"done","queue_duration_s":1.384083,"scheduling_latency_s":1.384106,"redis_calls":13,"redis_duration_s":0.002751,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000831,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00192,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010618,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015685,"completed_at":"2026-09-10T19:32:07.702Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.700275] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.703Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db4ea68a15b6679096fb27f9","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db4ea68a15b6679096fb27f9: start","job_status":"start","queue_duration_s":1.401195,"scheduling_latency_s":1.401218} gitlab # [ 172.701839] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.768Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db4ea68a15b6679096fb27f9","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db4ea68a15b6679096fb27f9: done: 0.065358 sec","job_status":"done","queue_duration_s":1.401195,"scheduling_latency_s":1.401218,"redis_calls":13,"redis_duration_s":0.050664,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001888,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.048776,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.065358,"completed_at":"2026-09-10T19:32:07.768Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.812956] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.770Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"502c047b87334a6e126a2095","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-502c047b87334a6e126a2095: start","job_status":"start","queue_duration_s":1.468217,"scheduling_latency_s":1.468244} gitlab # [ 172.814371] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.789Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"502c047b87334a6e126a2095","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-502c047b87334a6e126a2095: done: 0.018462 sec","job_status":"done","queue_duration_s":1.468217,"scheduling_latency_s":1.468244,"redis_calls":13,"redis_duration_s":0.003849,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001989,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00186,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011156,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018462,"completed_at":"2026-09-10T19:32:07.788Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.884718] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.800Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6548e4e1df2d64ec9a027aa6","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6548e4e1df2d64ec9a027aa6: start","job_status":"start","queue_duration_s":1.498083,"scheduling_latency_s":1.498107} gitlab # [ 172.885905] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.816Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6548e4e1df2d64ec9a027aa6","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6548e4e1df2d64ec9a027aa6: done: 0.015697 sec","job_status":"done","queue_duration_s":1.498083,"scheduling_latency_s":1.498107,"redis_calls":13,"redis_duration_s":0.002626,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000793,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001833,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010577,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015697,"completed_at":"2026-09-10T19:32:07.816Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.960314] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.817Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92b722433274a07f1731cc5c","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92b722433274a07f1731cc5c: start","job_status":"start","queue_duration_s":1.51524,"scheduling_latency_s":1.515261} gitlab # [ 172.961847] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.832Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92b722433274a07f1731cc5c","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92b722433274a07f1731cc5c: done: 0.015024 sec","job_status":"done","queue_duration_s":1.51524,"scheduling_latency_s":1.515261,"redis_calls":13,"redis_duration_s":0.002667,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000782,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001885,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01035,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015024,"completed_at":"2026-09-10T19:32:07.832Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.052190] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.833Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d737e408d40d30a465329379","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d737e408d40d30a465329379: start","job_status":"start","queue_duration_s":1.531585,"scheduling_latency_s":1.531605} gitlab # [ 173.053476] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.849Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d737e408d40d30a465329379","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d737e408d40d30a465329379: done: 0.015085 sec","job_status":"done","queue_duration_s":1.531585,"scheduling_latency_s":1.531605,"redis_calls":13,"redis_duration_s":0.002618,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000832,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001786,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010293,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015085,"completed_at":"2026-09-10T19:32:07.848Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.127419] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.850Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ff3ccadb64a1d11b615ba07","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ff3ccadb64a1d11b615ba07: start","job_status":"start","queue_duration_s":1.547913,"scheduling_latency_s":1.547934} gitlab # [ 173.128297] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.865Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ff3ccadb64a1d11b615ba07","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ff3ccadb64a1d11b615ba07: done: 0.015335 sec","job_status":"done","queue_duration_s":1.547913,"scheduling_latency_s":1.547934,"redis_calls":13,"redis_duration_s":0.002852,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000836,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002016,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010318,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015335,"completed_at":"2026-09-10T19:32:07.865Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.149149] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.866Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9459ceaf66e7c0d5fb57bb73","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9459ceaf66e7c0d5fb57bb73: start","job_status":"start","queue_duration_s":1.564577,"scheduling_latency_s":1.564599} gitlab # [ 173.168426] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.882Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9459ceaf66e7c0d5fb57bb73","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9459ceaf66e7c0d5fb57bb73: done: 0.015379 sec","job_status":"done","queue_duration_s":1.564577,"scheduling_latency_s":1.564599,"redis_calls":13,"redis_duration_s":0.002869,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000839,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00203,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010184,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015379,"completed_at":"2026-09-10T19:32:07.882Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.229907] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.883Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"723d618ca9d93667d73e1ac1","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-723d618ca9d93667d73e1ac1: start","job_status":"start","queue_duration_s":1.581197,"scheduling_latency_s":1.581217} gitlab # [ 173.299350] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.898Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"723d618ca9d93667d73e1ac1","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-723d618ca9d93667d73e1ac1: done: 0.015118 sec","job_status":"done","queue_duration_s":1.581197,"scheduling_latency_s":1.581217,"redis_calls":13,"redis_duration_s":0.002657,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000827,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00183,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010145,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015118,"completed_at":"2026-09-10T19:32:07.898Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.308319] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.900Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"434514f73c0f119412860a4f","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-434514f73c0f119412860a4f: start","job_status":"start","queue_duration_s":1.597647,"scheduling_latency_s":1.597667} gitlab # [ 173.386367] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.915Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"434514f73c0f119412860a4f","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-434514f73c0f119412860a4f: done: 0.015121 sec","job_status":"done","queue_duration_s":1.597647,"scheduling_latency_s":1.597667,"redis_calls":13,"redis_duration_s":0.0027029999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00082,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001883,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010142,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015121,"completed_at":"2026-09-10T19:32:07.915Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.387726] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.916Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6f53b45ac994033be8a86ee7","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6f53b45ac994033be8a86ee7: start","job_status":"start","queue_duration_s":1.614086,"scheduling_latency_s":1.614105} gitlab # [ 173.406931] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.931Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6f53b45ac994033be8a86ee7","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6f53b45ac994033be8a86ee7: done: 0.015052 sec","job_status":"done","queue_duration_s":1.614086,"scheduling_latency_s":1.614105,"redis_calls":13,"redis_duration_s":0.002738,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000989,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001749,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010224,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015052,"completed_at":"2026-09-10T19:32:07.931Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.487049] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.932Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6088c142b72fb60e439e418c","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6088c142b72fb60e439e418c: start","job_status":"start","queue_duration_s":1.630605,"scheduling_latency_s":1.630627} gitlab # [ 173.488346] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.947Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6088c142b72fb60e439e418c","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6088c142b72fb60e439e418c: done: 0.015018 sec","job_status":"done","queue_duration_s":1.630605,"scheduling_latency_s":1.630627,"redis_calls":13,"redis_duration_s":0.0026160000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000778,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001838,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010179,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015018,"completed_at":"2026-09-10T19:32:07.947Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.570107] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.949Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a38f9826b8c3f07bde644b19","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a38f9826b8c3f07bde644b19: start","job_status":"start","queue_duration_s":1.646996,"scheduling_latency_s":1.647018} gitlab # [ 173.571489] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.964Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a38f9826b8c3f07bde644b19","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a38f9826b8c3f07bde644b19: done: 0.015192 sec","job_status":"done","queue_duration_s":1.646996,"scheduling_latency_s":1.647018,"redis_calls":13,"redis_duration_s":0.00279,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000836,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001954,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010357,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015192,"completed_at":"2026-09-10T19:32:07.964Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.625237] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.966Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"82d1d36d738b4d7888cdf8ea","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-82d1d36d738b4d7888cdf8ea: start","job_status":"start","queue_duration_s":1.66362,"scheduling_latency_s":1.66364} gitlab # [ 173.644475] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.980Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"82d1d36d738b4d7888cdf8ea","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-82d1d36d738b4d7888cdf8ea: done: 0.014988 sec","job_status":"done","queue_duration_s":1.66362,"scheduling_latency_s":1.66364,"redis_calls":13,"redis_duration_s":0.0026680000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000831,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001837,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010316,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014988,"completed_at":"2026-09-10T19:32:07.980Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.723954] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.982Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d1e09241d03ec6005fc035a","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d1e09241d03ec6005fc035a: start","job_status":"start","queue_duration_s":1.68002,"scheduling_latency_s":1.680041} gitlab # [ 173.725092] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d1e09241d03ec6005fc035a","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d1e09241d03ec6005fc035a: done: 0.015095 sec","job_status":"done","queue_duration_s":1.68002,"scheduling_latency_s":1.680041,"redis_calls":13,"redis_duration_s":0.002556,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000833,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001723,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010415,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015095,"completed_at":"2026-09-10T19:32:07.997Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.797923] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:07.998Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0db5c7fb9b8d8c779c98fa09","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0db5c7fb9b8d8c779c98fa09: start","job_status":"start","queue_duration_s":1.696568,"scheduling_latency_s":1.696589} gitlab # [ 173.870290] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.013Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0db5c7fb9b8d8c779c98fa09","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0db5c7fb9b8d8c779c98fa09: done: 0.015053 sec","job_status":"done","queue_duration_s":1.696568,"scheduling_latency_s":1.696589,"redis_calls":13,"redis_duration_s":0.002639,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000796,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001843,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010345,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015053,"completed_at":"2026-09-10T19:32:08.013Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.872136] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.015Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f55bb80b44cfe2e7e39922e","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f55bb80b44cfe2e7e39922e: start","job_status":"start","queue_duration_s":1.712876,"scheduling_latency_s":1.712897} gitlab # [ 173.884276] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.030Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f55bb80b44cfe2e7e39922e","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f55bb80b44cfe2e7e39922e: done: 0.015172 sec","job_status":"done","queue_duration_s":1.712876,"scheduling_latency_s":1.712897,"redis_calls":13,"redis_duration_s":0.002687,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000826,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001861,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01036,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015172,"completed_at":"2026-09-10T19:32:08.030Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.957439] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.031Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3eca13c7852e2842d2f5d5d3","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3eca13c7852e2842d2f5d5d3: start","job_status":"start","queue_duration_s":1.729351,"scheduling_latency_s":1.729374} gitlab # [ 174.028377] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.046Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3eca13c7852e2842d2f5d5d3","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3eca13c7852e2842d2f5d5d3: done: 0.01518 sec","job_status":"done","queue_duration_s":1.729351,"scheduling_latency_s":1.729374,"redis_calls":13,"redis_duration_s":0.002587,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000823,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001764,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010337,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01518,"completed_at":"2026-09-10T19:32:08.046Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.030159] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.048Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b037261f4954e3d02d178be","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b037261f4954e3d02d178be: start","job_status":"start","queue_duration_s":1.745762,"scheduling_latency_s":1.745783} gitlab # [ 174.042874] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.063Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b037261f4954e3d02d178be","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b037261f4954e3d02d178be: done: 0.01529 sec","job_status":"done","queue_duration_s":1.745762,"scheduling_latency_s":1.745783,"redis_calls":13,"redis_duration_s":0.002917,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001158,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001759,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010143,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01529,"completed_at":"2026-09-10T19:32:08.063Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.147292] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.064Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4991c490868b1e4db66827bf","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4991c490868b1e4db66827bf: start","job_status":"start","queue_duration_s":1.762294,"scheduling_latency_s":1.762349} gitlab # [ 174.148384] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.079Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4991c490868b1e4db66827bf","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4991c490868b1e4db66827bf: done: 0.015251 sec","job_status":"done","queue_duration_s":1.762294,"scheduling_latency_s":1.762349,"redis_calls":13,"redis_duration_s":0.002691,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000874,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001817,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010205,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015251,"completed_at":"2026-09-10T19:32:08.079Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.186914] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.081Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4586685ed85f931bd9e4934f","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4586685ed85f931bd9e4934f: start","job_status":"start","queue_duration_s":1.778812,"scheduling_latency_s":1.778833} gitlab # [ 174.264526] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.096Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4586685ed85f931bd9e4934f","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4586685ed85f931bd9e4934f: done: 0.015109 sec","job_status":"done","queue_duration_s":1.778812,"scheduling_latency_s":1.778833,"redis_calls":13,"redis_duration_s":0.002592,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000824,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001768,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010196,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015109,"completed_at":"2026-09-10T19:32:08.096Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.267996] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.097Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b45906c5b7297163e6eaa964","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b45906c5b7297163e6eaa964: start","job_status":"start","queue_duration_s":1.795195,"scheduling_latency_s":1.795216} gitlab # [ 174.288161] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.112Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b45906c5b7297163e6eaa964","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b45906c5b7297163e6eaa964: done: 0.015363 sec","job_status":"done","queue_duration_s":1.795195,"scheduling_latency_s":1.795216,"redis_calls":13,"redis_duration_s":0.0025830000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000776,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001807,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010242,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015363,"completed_at":"2026-09-10T19:32:08.112Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.351263] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.114Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea117fa4d696f07ae83fd5ba","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea117fa4d696f07ae83fd5ba: start","job_status":"start","queue_duration_s":1.812017,"scheduling_latency_s":1.81204} gitlab # [ 174.437478] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.129Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea117fa4d696f07ae83fd5ba","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea117fa4d696f07ae83fd5ba: done: 0.015157 sec","job_status":"done","queue_duration_s":1.812017,"scheduling_latency_s":1.81204,"redis_calls":13,"redis_duration_s":0.002532,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000795,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001737,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010352,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015157,"completed_at":"2026-09-10T19:32:08.129Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.439301] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.131Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d24b9a88d6e1b35455ac4d05","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d24b9a88d6e1b35455ac4d05: start","job_status":"start","queue_duration_s":1.828621,"scheduling_latency_s":1.828641} gitlab # [ 174.452200] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.146Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d24b9a88d6e1b35455ac4d05","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d24b9a88d6e1b35455ac4d05: done: 0.015066 sec","job_status":"done","queue_duration_s":1.828621,"scheduling_latency_s":1.828641,"redis_calls":13,"redis_duration_s":0.002673,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000821,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001852,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01039,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015066,"completed_at":"2026-09-10T19:32:08.145Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.531448] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.147Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f3888616b687290bc018f250","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f3888616b687290bc018f250: start","job_status":"start","queue_duration_s":1.845095,"scheduling_latency_s":1.845116} gitlab # [ 174.533425] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.162Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f3888616b687290bc018f250","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f3888616b687290bc018f250: done: 0.015328 sec","job_status":"done","queue_duration_s":1.845095,"scheduling_latency_s":1.845116,"redis_calls":13,"redis_duration_s":0.002702,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00078,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001922,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010474,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015328,"completed_at":"2026-09-10T19:32:08.162Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.638073] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.164Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"555825cc5026578acf422824","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-555825cc5026578acf422824: start","job_status":"start","queue_duration_s":1.86186,"scheduling_latency_s":1.86188} gitlab # [ 174.639498] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.179Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"555825cc5026578acf422824","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-555825cc5026578acf422824: done: 0.015152 sec","job_status":"done","queue_duration_s":1.86186,"scheduling_latency_s":1.86188,"redis_calls":13,"redis_duration_s":0.002659,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000863,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001796,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010438,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015152,"completed_at":"2026-09-10T19:32:08.179Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.724959] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.180Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4cf574442ccf6cc27febcf77","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4cf574442ccf6cc27febcf77: start","job_status":"start","queue_duration_s":1.878519,"scheduling_latency_s":1.87854} gitlab # [ 174.726325] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.195Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4cf574442ccf6cc27febcf77","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4cf574442ccf6cc27febcf77: done: 0.015008 sec","job_status":"done","queue_duration_s":1.878519,"scheduling_latency_s":1.87854,"redis_calls":13,"redis_duration_s":0.002587,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000838,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001749,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010378,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015008,"completed_at":"2026-09-10T19:32:08.195Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.795873] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.197Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b68f800b63990bd2e984e1ab","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b68f800b63990bd2e984e1ab: start","job_status":"start","queue_duration_s":1.8948,"scheduling_latency_s":1.894823} gitlab # [ 174.797067] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.212Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b68f800b63990bd2e984e1ab","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b68f800b63990bd2e984e1ab: done: 0.015171 sec","job_status":"done","queue_duration_s":1.8948,"scheduling_latency_s":1.894823,"redis_calls":13,"redis_duration_s":0.002613,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000831,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001782,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010355,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015171,"completed_at":"2026-09-10T19:32:08.212Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.871081] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.213Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b13bf1249140241df0d00645","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b13bf1249140241df0d00645: start","job_status":"start","queue_duration_s":1.911235,"scheduling_latency_s":1.911255} gitlab # [ 174.872441] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.228Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b13bf1249140241df0d00645","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b13bf1249140241df0d00645: done: 0.015216 sec","job_status":"done","queue_duration_s":1.911235,"scheduling_latency_s":1.911255,"redis_calls":13,"redis_duration_s":0.00279,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000832,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001958,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01019,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015216,"completed_at":"2026-09-10T19:32:08.228Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.943148] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.230Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e8a5950039b0a7fc0ba84584","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e8a5950039b0a7fc0ba84584: start","job_status":"start","queue_duration_s":1.927757,"scheduling_latency_s":1.927777} gitlab # [ 174.944526] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.245Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e8a5950039b0a7fc0ba84584","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e8a5950039b0a7fc0ba84584: done: 0.015286 sec","job_status":"done","queue_duration_s":1.927757,"scheduling_latency_s":1.927777,"redis_calls":13,"redis_duration_s":0.002492,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00078,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001712,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010299,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015286,"completed_at":"2026-09-10T19:32:08.245Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.029307] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.246Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c4c97be0ccf2de327f90a91c","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c4c97be0ccf2de327f90a91c: start","job_status":"start","queue_duration_s":1.944395,"scheduling_latency_s":1.944415} gitlab # [ 175.030447] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.261Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c4c97be0ccf2de327f90a91c","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c4c97be0ccf2de327f90a91c: done: 0.015188 sec","job_status":"done","queue_duration_s":1.944395,"scheduling_latency_s":1.944415,"redis_calls":13,"redis_duration_s":0.002555,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000796,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001759,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010221,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015188,"completed_at":"2026-09-10T19:32:08.261Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.100895] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.263Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d76cb007611353b07e8049f","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d76cb007611353b07e8049f: start","job_status":"start","queue_duration_s":1.960866,"scheduling_latency_s":1.960886} gitlab # [ 175.102107] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.278Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d76cb007611353b07e8049f","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d76cb007611353b07e8049f: done: 0.015038 sec","job_status":"done","queue_duration_s":1.960866,"scheduling_latency_s":1.960886,"redis_calls":13,"redis_duration_s":0.002613,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000821,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001792,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010216,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015038,"completed_at":"2026-09-10T19:32:08.278Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.173916] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.279Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4162ac5d77411ca4cfcb77bf","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4162ac5d77411ca4cfcb77bf: start","job_status":"start","queue_duration_s":1.977469,"scheduling_latency_s":1.977491} gitlab # [ 175.174741] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.294Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4162ac5d77411ca4cfcb77bf","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4162ac5d77411ca4cfcb77bf: done: 0.015023 sec","job_status":"done","queue_duration_s":1.977469,"scheduling_latency_s":1.977491,"redis_calls":13,"redis_duration_s":0.002528,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000789,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001739,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010221,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015023,"completed_at":"2026-09-10T19:32:08.294Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.264073] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.296Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a670e4b4255bab8c40fdf1a6","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a670e4b4255bab8c40fdf1a6: start","job_status":"start","queue_duration_s":1.993887,"scheduling_latency_s":1.993908} gitlab # [ 175.265496] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.311Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a670e4b4255bab8c40fdf1a6","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a670e4b4255bab8c40fdf1a6: done: 0.015029 sec","job_status":"done","queue_duration_s":1.993887,"scheduling_latency_s":1.993908,"redis_calls":13,"redis_duration_s":0.0026379999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000828,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00181,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010338,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015029,"completed_at":"2026-09-10T19:32:08.311Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.355087] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.312Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"897b20edf981d6b96b032083","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-897b20edf981d6b96b032083: start","job_status":"start","queue_duration_s":2.010368,"scheduling_latency_s":2.010389} gitlab # [ 175.356431] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.327Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"897b20edf981d6b96b032083","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-897b20edf981d6b96b032083: done: 0.015083 sec","job_status":"done","queue_duration_s":2.010368,"scheduling_latency_s":2.010389,"redis_calls":13,"redis_duration_s":0.00258,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000822,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001758,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010402,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015083,"completed_at":"2026-09-10T19:32:08.327Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.428978] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.329Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c73f1419e900b67afe471c39","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c73f1419e900b67afe471c39: start","job_status":"start","queue_duration_s":2.02684,"scheduling_latency_s":2.026861} gitlab # [ 175.430350] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.344Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c73f1419e900b67afe471c39","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c73f1419e900b67afe471c39: done: 0.01507 sec","job_status":"done","queue_duration_s":2.02684,"scheduling_latency_s":2.026861,"redis_calls":13,"redis_duration_s":0.002817,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000825,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001992,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010402,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01507,"completed_at":"2026-09-10T19:32:08.344Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.501784] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.345Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2cdcef2e04457f427d698c8","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2cdcef2e04457f427d698c8: start","job_status":"start","queue_duration_s":2.043361,"scheduling_latency_s":2.043381} gitlab # [ 175.503202] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.360Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2cdcef2e04457f427d698c8","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2cdcef2e04457f427d698c8: done: 0.014957 sec","job_status":"done","queue_duration_s":2.043361,"scheduling_latency_s":2.043381,"redis_calls":13,"redis_duration_s":0.002624,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00082,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001804,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010319,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014957,"completed_at":"2026-09-10T19:32:08.360Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.597260] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.362Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a624bf3133d062de7ad961aa","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a624bf3133d062de7ad961aa: start","job_status":"start","queue_duration_s":2.059803,"scheduling_latency_s":2.059825} gitlab # [ 175.598902] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.377Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a624bf3133d062de7ad961aa","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a624bf3133d062de7ad961aa: done: 0.01531 sec","job_status":"done","queue_duration_s":2.059803,"scheduling_latency_s":2.059825,"redis_calls":13,"redis_duration_s":0.002615,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000812,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001803,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010494,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01531,"completed_at":"2026-09-10T19:32:08.377Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.668063] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.378Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9e725bccb3ea9c803aa352f","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9e725bccb3ea9c803aa352f: start","job_status":"start","queue_duration_s":2.076483,"scheduling_latency_s":2.076504} gitlab # [ 175.669189] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:08.393Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9e725bccb3ea9c803aa352f","created_at":"2026-09-10T19:32:06.224Z","trace_propagation_headers":{"sentry-trace":"144ca6288d7b4d85892b703cfdcca842-88757f71daf24e3f","baggage":"sentry-trace_id=144ca6288d7b4d85892b703cfdcca842,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6e120f9b68520020287ca97cd2cada16","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:06.302Z","job_size_bytes":2,"pid":1676,"sidekiq_tid":"majc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9e725bccb3ea9c803aa352f: done: 0.015154 sec","job_status":"done","queue_duration_s":2.076483,"scheduling_latency_s":2.076504,"redis_calls":13,"redis_duration_s":0.002635,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000827,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001808,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010337,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015154,"completed_at":"2026-09-10T19:32:08.393Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.050732] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 6245239229 wd_nsec: 6245238898 gitlab # [ 176.004934] setup-gitlab-start[2258]: W, [2026-09-10T19:32:15.761006 #2258] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 199.425791] postgres[2499]: [2499] ERROR: relation "table_without_models" does not exist gitlab # [ 199.430429] postgres[2499]: [2499] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 202.588861] runuser[2257]: pam_unix(runuser:session): session closed for user gitlab gitlab # [ 202.593929] setup-gitlab-start[2228]: + systemctl restart gitlab-sidekiq.service gitlab # [ 202.615705] systemd[1]: Stopping gitlab-sidekiq.service... gitlab # [ 202.620486] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:42.371Z","message":"Shutting down"} gitlab # [ 202.623221] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:42.375Z","message":"Scheduler exiting..."} gitlab # [ 202.625844] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:42.375Z","message":"Terminating quiet threads for default capsule"} gitlab # [ 202.629317] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:42.376Z","message":"Scheduler exiting..."} gitlab # [ 203.121468] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:42.877Z","message":"Pausing to allow jobs to finish..."} gitlab # [ 204.124989] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:43.880Z","message":"Bye!"} gitlab # [ 204.162584] sidekiq-cluster[1676]: {"severity":"INFO","time":"2026-09-10T19:32:43.918Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1676,"worker_id":"sidekiq_0","memwd_rss_bytes":969375744,"retry":0} gitlab # [ 205.017913] systemd[1]: gitlab-sidekiq.service: Deactivated successfully. gitlab # [ 205.022395] systemd[1]: Stopped gitlab-sidekiq.service. gitlab # [ 205.026222] systemd[1]: gitlab-sidekiq.service: Consumed 1min 1.357s CPU time over 2min 5.253s wall clock time, 972.6M memory peak, 4.2M read from disk. gitlab # [ 205.034853] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 205.039258] setup-gitlab-start[2228]: + mkdir -p /var/lib/secrets gitlab # [ 205.043241] setup-gitlab-start[2228]: + echo nixbot-test-token-of-20-chars gitlab # [ 205.047497] setup-gitlab-start[2228]: + chmod 644 /var/lib/secrets/gitlab-token gitlab # [ 205.051933] setup-gitlab-start[2228]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars gitlab # [ 205.324764] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:32:45 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 244 gitlab # [ 205.852750] gitaly[931]: time="2026-09-10T19:32:45.608Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M26CXSXMCMZSAKRK9V7RXHGR error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-10T19:32:55.007" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:32:45.608" pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.892927] gitaly[931]: time="2026-09-10T19:32:45.608Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M26CXSXMCMZSAKRK9V7RXHGR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-10T19:32:55.007" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:32:45.608" grpc.time_ms=0.279 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.994851] gitaly[931]: time="2026-09-10T19:32:45.751Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M26CXSXMCMZSAKRK9V7RXHGR error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-10T19:32:55.150" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:32:45.751" pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 205.997887] gitaly[931]: time="2026-09-10T19:32:45.751Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M26CXSXMCMZSAKRK9V7RXHGR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-10T19:32:55.150" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:32:45.751" grpc.time_ms=0.238 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.119908] gitaly[931]: time="2026-09-10T19:32:45.876Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M26CXSXMCMZSAKRK9V7RXHGR diskcache=2d9d87e6-4570-4704-a9df-014fe3503c03 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-10T19:32:55.216" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:32:45.818" pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.122967] gitaly[931]: time="2026-09-10T19:32:45.876Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=19 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1583 command.oublock=32 command.real_time_ms=27 command.spawn_token_fork_ms=1 command.system_time_ms=15 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M26CXSXMCMZSAKRK9V7RXHGR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-10T19:32:55.216" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:32:45.818" grpc.time_ms=57.956 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.188494] gitaly[931]: time="2026-09-10T19:32:45.944Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M26CXSXMCMZSAKRK9V7RXHGR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-10T19:32:55.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:32:45.944" pid=931 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 206.190418] gitaly[931]: time="2026-09-10T19:32:45.945Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M26CXSXMCMZSAKRK9V7RXHGR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-10T19:32:55.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:32:45.944" grpc.time_ms=0.723 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.210813] gitaly[931]: time="2026-09-10T19:32:45.967Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M26CXSXMCMZSAKRK9V7RXHGR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-10T19:32:55.012" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:32:45.953" grpc.time_ms=13.588 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.272794] gitaly[931]: time="2026-09-10T19:32:46.029Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M26CXSXMCMZSAKRK9V7RXHGR error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-10T19:32:56.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:32:46.028" pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.273974] gitaly[931]: time="2026-09-10T19:32:46.029Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M26CXSXMCMZSAKRK9V7RXHGR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-10T19:32:56.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:32:46.028" grpc.time_ms=0.243 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.346106] gitaly[931]: time="2026-09-10T19:32:46.102Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M26CXSXMCMZSAKRK9V7RXHGR diskcache=78ca9fd8-60bd-46da-8f77-f6cdbb760031 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-10T19:32:56.044" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:32:46.065" pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.347297] gitaly[931]: time="2026-09-10T19:32:46.103Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1582 command.oublock=32 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CXSXMCMZSAKRK9V7RXHGR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-10T19:32:56.044" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:32:46.065" grpc.time_ms=37.519 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.356495] gitaly[931]: time="2026-09-10T19:32:46.112Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M26CXSXMCMZSAKRK9V7RXHGR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-10T19:32:56.091" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:32:46.112" pid=931 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 206.357712] gitaly[931]: time="2026-09-10T19:32:46.112Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M26CXSXMCMZSAKRK9V7RXHGR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-10T19:32:56.091" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:32:46.112" grpc.time_ms=0.449 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.369730] gitaly[931]: time="2026-09-10T19:32:46.126Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CXSXMCMZSAKRK9V7RXHGR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-10T19:32:56.097" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:32:46.118" grpc.time_ms=6.99 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.399727] gitaly[931]: time="2026-09-10T19:32:46.155Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M26CXSXMCMZSAKRK9V7RXHGR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-10T19:32:56.134" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:32:46.155" grpc.time_ms=0.269 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.405248] gitaly[931]: time="2026-09-10T19:32:46.161Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M26CXSXMCMZSAKRK9V7RXHGR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-10T19:32:56.140" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:32:46.160" grpc.time_ms=0.37 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.965946] gitaly[931]: time="2026-09-10T19:32:46.721Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=618 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CXSXMCMZSAKRK9V7RXHGR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-10T19:32:56.114" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:32:46.715" grpc.time_ms=6.382 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.237540] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:32:46 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 1884 gitlab # [ 207.246534] setup-gitlab-start[2553]: {"id":1,"message":null,"error":null} gitlab # [ 208.395717] sidekiq-cluster[2537]: {"severity":"INFO","time":"2026-09-10T19:32:48.133Z","message":"Starting cluster with 1 processes"} gitlab # [ 212.398808] gitaly[931]: time="2026-09-10T19:32:52.154Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CY0P2PJFG35N9TX1VMP5C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-10T19:33:02.124" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:32:52.144" grpc.time_ms=9.441 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 212.450738] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:32:52 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 167 gitlab # [ 212.454532] setup-gitlab-start[2228]: + mkdir -p /tmp/test-flake gitlab # [ 212.456331] setup-gitlab-start[2228]: + cd /tmp/test-flake gitlab # [ 212.458253] setup-gitlab-start[2228]: + git init -b master gitlab # [ 212.475381] setup-gitlab-start[2626]: Initialized empty Git repository in /tmp/test-flake/.git/ gitlab # [ 212.477448] setup-gitlab-start[2228]: + git config user.name test gitlab # [ 212.479100] setup-gitlab-start[2228]: + git config user.email test@example.com gitlab # [ 212.480937] setup-gitlab-start[2228]: + cat gitlab # [ 212.482176] setup-gitlab-start[2228]: + git add flake.nix gitlab # [ 212.486616] setup-gitlab-start[2228]: + git commit -m 'initial commit' gitlab # [ 212.506254] setup-gitlab-start[2631]: [master (root-commit) b2555a8] initial commit gitlab # [ 212.507270] setup-gitlab-start[2631]: 1 file changed, 10 insertions(+) gitlab # [ 212.508096] setup-gitlab-start[2631]: create mode 100644 flake.nix gitlab # [ 212.508939] setup-gitlab-start[2228]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git gitlab # [ 212.519478] setup-gitlab-start[2228]: + git push -u origin master gitlab # [ 212.575226] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:32:52 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 34 gitlab # [ 212.685694] gitaly[931]: time="2026-09-10T19:32:52.441Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M26CY0ZN3ESB6HJQTNN93A6C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:32:52.429" grpc.time_ms=11.458 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 212.689176] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:32:52 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 101 gitlab # [ 212.987043] gitaly[931]: time="2026-09-10T19:32:52.743Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-10T19:33:02.129" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:32:52.730" grpc.time_ms=12.309 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.085662] gitaly[931]: time="2026-09-10T19:32:52.842Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=331 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-10T19:33:22.024" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-10T19:32:52.825" grpc.time_ms=16.431 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.163752] gitaly[931]: time="2026-09-10T19:32:52.920Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M26CY14TA2KV1CF07DG86K3P duration_ms=254ns method=POST pid=931 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 213.212194] gitaly[931]: time="2026-09-10T19:32:52.968Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M26CY14TA2KV1CF07DG86K3P duration_ms=36ns method=POST pid=931 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 213.214179] gitaly[931]: time="2026-09-10T19:32:52.969Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-10T19:32:52.664" grpc.time_ms=304.587 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 213.307986] gitaly[931]: time="2026-09-10T19:32:53.064Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-10T19:32:53.063" grpc.time_ms=0.861 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 213.358988] gitaly[931]: time="2026-09-10T19:32:53.115Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-10T19:32:53.113" grpc.time_ms=1.142 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 213.402790] gitaly[931]: time="2026-09-10T19:32:53.158Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-10T19:32:53.158" grpc.time_ms=0.366 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 213.706171] gitaly[931]: time="2026-09-10T19:32:53.462Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-10T19:33:03.005" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:32:53.455" grpc.time_ms=6.748 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.806898] gitaly[931]: time="2026-09-10T19:32:53.563Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1569 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-10T19:33:03.094" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:32:53.545" grpc.time_ms=17.897 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.838445] gitaly[931]: time="2026-09-10T19:32:53.594Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M26CY14TA2KV1CF07DG86K3P duration_ms=398ns method=POST pid=931 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 213.840357] gitaly[931]: time="2026-09-10T19:32:53.595Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-10T19:32:53.196" grpc.time_ms=399.146 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 213.874727] gitaly[931]: time="2026-09-10T19:32:53.631Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P diskcache=706186d3-cd71-49b7-a7e7-8a070c8fbd13 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:32:52.579" pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 213.875834] gitaly[931]: time="2026-09-10T19:32:53.631Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=269 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=892928 command.minflt=13345 command.oublock=32 command.real_time_ms=1043 command.spawn_token_fork_ms=0 command.system_time_ms=198 command.user_time_ms=71 component=gitaly.StreamServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:32:52.579" grpc.time_ms=1051.779 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 213.877269] gitlab-workhorse[830]: time="2026-09-10T19:32:53Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 213.877968] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:32:53 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 1125 gitlab # [ 213.917844] setup-gitlab-start[2635]: To http://gitlab/root/test-flake.git gitlab # [ 213.918969] setup-gitlab-start[2635]: * [new branch] master -> master gitlab # [ 213.923149] setup-gitlab-start[2635]: branch 'master' set up to track 'origin/master'. gitlab # [ 213.927139] systemd[1]: Finished setup-gitlab.service. gitlab # [ 213.931571] systemd[1]: Starting nixbot CI service... gitlab: (finished: waiting for unit setup-gitlab.service, in 215.44 seconds) gitlab: waiting for unit nixbot.service gitlab # [ 215.047754] nixbot[2723]: {"ts": "2026-09-10T19:32:54.803Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} gitlab # [ 215.087986] nixbot[2723]: {"ts": "2026-09-10T19:32:54.844Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} gitlab # [ 215.123633] nixbot[2723]: {"ts": "2026-09-10T19:32:54.879Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} gitlab # [ 215.134407] nixbot[2723]: {"ts": "2026-09-10T19:32:54.890Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} gitlab # [ 215.143706] nixbot[2723]: {"ts": "2026-09-10T19:32:54.900Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} gitlab # [ 215.153224] nixbot[2723]: {"ts": "2026-09-10T19:32:54.909Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} gitlab # [ 215.160374] nixbot[2723]: {"ts": "2026-09-10T19:32:54.916Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} gitlab # [ 215.167672] nixbot[2723]: {"ts": "2026-09-10T19:32:54.923Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} gitlab # [ 215.176819] nixbot[2723]: {"ts": "2026-09-10T19:32:54.933Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} gitlab # [ 215.183207] nixbot[2723]: {"ts": "2026-09-10T19:32:54.939Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} gitlab # [ 215.194972] nixbot[2723]: {"ts": "2026-09-10T19:32:54.951Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} gitlab # [ 215.207415] nixbot[2723]: {"ts": "2026-09-10T19:32:54.963Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} gitlab # [ 215.215767] nixbot[2723]: {"ts": "2026-09-10T19:32:54.972Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} gitlab # [ 215.222827] nixbot[2723]: {"ts": "2026-09-10T19:32:54.979Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} gitlab # [ 215.233400] nixbot[2723]: {"ts": "2026-09-10T19:32:54.989Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} gitlab # [ 215.240185] nixbot[2723]: {"ts": "2026-09-10T19:32:54.996Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} gitlab # [ 215.247668] nixbot[2723]: {"ts": "2026-09-10T19:32:55.003Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} gitlab # [ 215.251476] nixbot[2723]: {"ts": "2026-09-10T19:32:55.007Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} gitlab # [ 215.259103] nixbot[2723]: {"ts": "2026-09-10T19:32:55.015Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} gitlab # [ 215.266264] nixbot[2723]: {"ts": "2026-09-10T19:32:55.022Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"} gitlab # [ 215.272712] nixbot[2723]: {"ts": "2026-09-10T19:32:55.028Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"} gitlab # [ 215.283737] nixbot[2723]: {"ts": "2026-09-10T19:32:55.040Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"} gitlab # [ 215.290516] nixbot[2723]: {"ts": "2026-09-10T19:32:55.046Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"} gitlab # [ 215.297307] nixbot[2723]: {"ts": "2026-09-10T19:32:55.053Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"} gitlab # [ 215.304452] nixbot[2723]: {"ts": "2026-09-10T19:32:55.060Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"} gitlab # [ 215.309427] nixbot[2723]: {"ts": "2026-09-10T19:32:55.065Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"} gitlab # [ 215.315930] nixbot[2723]: {"ts": "2026-09-10T19:32:55.072Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"} gitlab # [ 215.323849] nixbot[2723]: {"ts": "2026-09-10T19:32:55.080Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"} gitlab # [ 215.327799] nixbot[2723]: {"ts": "2026-09-10T19:32:55.084Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"} gitlab # [ 215.346262] nixbot[2723]: {"ts": "2026-09-10T19:32:55.102Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"} gitlab # [ 215.353563] nixbot[2723]: {"ts": "2026-09-10T19:32:55.109Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"} gitlab # [ 215.359808] nixbot[2723]: {"ts": "2026-09-10T19:32:55.116Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"} gitlab # [ 215.363669] nixbot[2723]: {"ts": "2026-09-10T19:32:55.119Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"} gitlab # [ 215.371383] nixbot[2723]: {"ts": "2026-09-10T19:32:55.127Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"} gitlab # [ 215.781278] nixbot[2723]: INFO: Started server process [2723] gitlab # [ 215.784695] nixbot[2723]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit) gitlab # [ 215.809249] nixbot[2723]: INFO: ::ffff:127.0.0.1:34516 - "GET /health HTTP/1.1" 200 OK gitlab # [ 215.812736] systemd[1]: Started nixbot CI service. gitlab # [ 215.819453] systemd[1]: Reached target Multi-User System. gitlab # [ 215.823373] systemd[1]: Startup finished in 1.413s (kernel) + 3.628s (initrd) + 3min 30.780s (userspace) = 3min 35.822s. gitlab: (finished: waiting for unit nixbot.service, in 2.10 seconds) gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health gitlab # [ 216.660559] nixbot[2723]: INFO: ::ffff:127.0.0.1:34530 - "GET /health HTTP/1.1" 200 OK gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.04 seconds) subtest: project discovered and webhook registered gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 216.747085] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:32:56 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 48 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.09 seconds) gitlab # [ 216.789461] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:32:56 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 983 gitlab # [ 216.792101] nixbot[2723]: {"ts": "2026-09-10T19:32:56.547Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 216.821577] nixbot[2723]: {"ts": "2026-09-10T19:32:56.578Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} gitlab # [ 216.894134] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:32:56 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 51 gitlab # [ 216.896245] nixbot[2723]: {"ts": "2026-09-10T19:32:56.651Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 216.898070] nixbot[2723]: {"ts": "2026-09-10T19:32:56.652Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} gitlab # [ 217.126589] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:32:56 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 918 "" "python-httpx/0.28.1" 227 gitlab # [ 217.135504] nixbot[2723]: {"ts": "2026-09-10T19:32:56.891Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""} gitlab # [ 217.340477] nixbot[2723]: {"ts": "2026-09-10T19:32:57.096Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 217.350872] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:32:57 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 194 gitlab # [ 217.423161] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:32:57 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "python-httpx/0.28.1" 49 gitlab # [ 217.432511] nixbot[2723]: {"ts": "2026-09-10T19:32:57.188Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 217.508299] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:32:57 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 918 "" "python-httpx/0.28.1" 57 gitlab # [ 217.510949] nixbot[2723]: {"ts": "2026-09-10T19:32:57.262Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""} gitlab # [ 217.600424] gitaly[931]: time="2026-09-10T19:32:57.355Z" level=info msg="finished unary call with code OK" catfile.duration_ms=4 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=4 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M26CY5T7FZFPQ6BTXXHWN4CV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-09-10T19:33:27.035" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:32:57.336" grpc.time_ms=19.123 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.654094] gitaly[931]: time="2026-09-10T19:32:57.410Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CY5T7FZFPQ6BTXXHWN4CV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:33:27.070" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:32:57.370" grpc.time_ms=39.86 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.683300] gitaly[931]: time="2026-09-10T19:32:57.439Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=16 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CY5T7FZFPQ6BTXXHWN4CV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-09-10T19:33:07.003" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:32:57.414" grpc.time_ms=25.107 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.714770] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:32:57 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 182 gitlab # [ 217.717399] nixbot[2723]: {"ts": "2026-09-10T19:32:57.471Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 217.799833] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:32:57 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 75 gitlab # [ 217.809456] nixbot[2723]: {"ts": "2026-09-10T19:32:57.565Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 217.819679] nixbot[2723]: {"ts": "2026-09-10T19:32:57.576Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} gitlab # [ 217.851525] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:32:57 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "curl/8.21.0" 61 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.10 seconds) (finished: subtest: project discovered and webhook registered, in 1.19 seconds) subtest: push delivers the webhook gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails' gitlab # [ 217.907575] gitaly[931]: time="2026-09-10T19:32:57.664Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CY64D3RKP0J6206E0YA67 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:32:57.656" grpc.time_ms=6.873 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 217.928990] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:32:57 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 72 gitlab # [ 218.012902] gitaly[931]: time="2026-09-10T19:32:57.769Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M26CY66T1AWW0KW13QJQJY1M grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:32:57.762" pid=931 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc gitlab # [ 218.014611] gitaly[931]: time="2026-09-10T19:32:57.769Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M26CY66T1AWW0KW13QJQJY1M grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:32:57.762" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.018505] gitaly[931]: time="2026-09-10T19:32:57.769Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M26CY66T1AWW0KW13QJQJY1M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:32:57.762" grpc.time_ms=7.536 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.019211] gitlab-workhorse[830]: time="2026-09-10T19:32:57Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=113 gitlab # [ 218.019977] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:32:57 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 83 gitlab # [ 218.179928] gitaly[931]: time="2026-09-10T19:32:57.935Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M26CY69KFRB76QP8D80GKP31 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:32:57.909" pid=931 remote_ip="2001:db8:1::1" request_sha=1e712b0dd80b144f3ff08cf5a4791aa6e781edac response_bytes=223 span.kind=server system=grpc gitlab # [ 218.183379] gitaly[931]: time="2026-09-10T19:32:57.935Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1202 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=9 component=gitaly.UnaryServerInterceptor correlation_id=01M26CY69KFRB76QP8D80GKP31 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:32:57.909" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.185832] gitlab-workhorse[830]: time="2026-09-10T19:32:57Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=223 gitlab # [ 218.188477] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:32:57 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 160 gitlab # [ 218.190549] gitaly[931]: time="2026-09-10T19:32:57.936Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1202 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=9 component=gitaly.UnaryServerInterceptor correlation_id=01M26CY69KFRB76QP8D80GKP31 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:32:57.909" grpc.time_ms=26.706 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.323567] gitaly[931]: time="2026-09-10T19:32:58.079Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CY6HJZ3B0J0HTB3NZ3KEQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:32:58.072" grpc.time_ms=7.704 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.325357] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:32:58 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 45 gitlab # [ 218.415234] gitaly[931]: time="2026-09-10T19:32:58.171Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M26CY6KS827BR3X2YRRDPARM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:32:58.159" pid=931 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 218.431410] gitaly[931]: time="2026-09-10T19:32:58.171Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M26CY6KS827BR3X2YRRDPARM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:32:58.159" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.452393] gitaly[931]: time="2026-09-10T19:32:58.174Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M26CY6KS827BR3X2YRRDPARM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:32:58.159" grpc.time_ms=14.168 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.475818] gitlab-workhorse[830]: time="2026-09-10T19:32:58Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 218.479045] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:32:58 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 126 gitlab # [ 218.560423] gitaly[931]: time="2026-09-10T19:32:58.316Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CY6RSP2TA33HQNCT2HMV3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:32:58.307" grpc.time_ms=9.526 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.583181] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:32:58 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 52 gitlab # [ 218.641398] gitaly[931]: time="2026-09-10T19:32:58.396Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M26CY6TG6FJAZ0DT4116YB26 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:32:58.379" pid=931 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc gitlab # [ 218.666653] gitaly[931]: time="2026-09-10T19:32:58.396Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1158 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M26CY6TG6FJAZ0DT4116YB26 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:32:58.379" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.690171] gitlab-workhorse[830]: time="2026-09-10T19:32:58Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=209 gitlab # [ 218.693551] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:32:58 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 79 gitlab # [ 218.696925] gitaly[931]: time="2026-09-10T19:32:58.397Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1158 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M26CY6TG6FJAZ0DT4116YB26 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:32:58.379" grpc.time_ms=17.663 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.741459] gitaly[931]: time="2026-09-10T19:32:58.497Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CY6YMAKT9N9YGBJMVJJXH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:33:28.096" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:32:58.496" grpc.time_ms=0.652 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.790826] gitaly[931]: time="2026-09-10T19:32:58.547Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CY6YMAKT9N9YGBJMVJJXH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-10T19:33:28.138" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:32:58.538" grpc.time_ms=8.569 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.811118] gitaly[931]: time="2026-09-10T19:32:58.567Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CY6YMAKT9N9YGBJMVJJXH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-10T19:33:08.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:32:58.560" grpc.time_ms=6.69 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.825549] gitaly[931]: time="2026-09-10T19:32:58.582Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CY6YMAKT9N9YGBJMVJJXH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-10T19:33:08.025" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:32:58.576" grpc.time_ms=5.862 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.883645] gitaly[931]: time="2026-09-10T19:32:58.639Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M26CY6YMAKT9N9YGBJMVJJXH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-10T19:33:08.088" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:32:58.639" grpc.time_ms=0.207 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.913175] gitaly[931]: time="2026-09-10T19:32:58.669Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M26CY6YMAKT9N9YGBJMVJJXH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-10T19:33:08.113" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:32:58.663" grpc.time_ms=5.604 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.964374] gitaly[931]: time="2026-09-10T19:32:58.720Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M26CY6YMAKT9N9YGBJMVJJXH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-10T19:32:58.720" grpc.time_ms=0.536 pid=931 span.kind=server system=grpc gitlab # [ 219.000488] gitaly[931]: time="2026-09-10T19:32:58.756Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M26CY6YMAKT9N9YGBJMVJJXH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-10T19:32:58.755" grpc.time_ms=1.331 pid=931 span.kind=server system=grpc gitlab # [ 219.022212] gitaly[931]: time="2026-09-10T19:32:58.778Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M26CY6YMAKT9N9YGBJMVJJXH diskcache=492f4a26-bad7-4d40-a25d-662a37565f05 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-10T19:33:08.140" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:32:58.690" pid=931 ref=refs/keep-around/b2555a8ee53ad96300d2a6b920912389a68d95c2 remote_ip="2001:db8:1::1" revision=b2555a8ee53ad96300d2a6b920912389a68d95c2 span.kind=server system=grpc username=root gitlab # [ 219.023411] gitaly[931]: time="2026-09-10T19:32:58.778Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=74 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5444 command.oublock=8 command.real_time_ms=80 command.spawn_token_fork_ms=0 command.system_time_ms=57 command.user_time_ms=17 component=gitaly.UnaryServerInterceptor correlation_id=01M26CY6YMAKT9N9YGBJMVJJXH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-10T19:33:08.140" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:32:58.690" grpc.time_ms=88.024 pid=931 ref=refs/keep-around/b2555a8ee53ad96300d2a6b920912389a68d95c2 remote_ip="2001:db8:1::1" revision=b2555a8ee53ad96300d2a6b920912389a68d95c2 span.kind=server system=grpc username=root gitlab # [ 219.026949] gitaly[931]: time="2026-09-10T19:32:58.783Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CY6YMAKT9N9YGBJMVJJXH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:33:28.382" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:32:58.783" grpc.time_ms=0.313 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 219.138737] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:32:58 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/b2555a8ee53ad96300d2a6b920912389a68d95c2 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 442 gitlab # [ 219.143396] nixbot[2723]: {"ts": "2026-09-10T19:32:58.899Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/b2555a8ee53ad96300d2a6b920912389a68d95c2 \"HTTP/1.1 201 Created\""} gitlab # [ 219.155265] nixbot[2723]: {"ts": "2026-09-10T19:32:58.911Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-b2555a8ee53a-0"} gitlab # [ 219.309430] nixbot[2723]: {"ts": "2026-09-10T19:32:59.065Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-b2555a8ee53a-0", "argv0": "bwrap"} gitlab # [ 219.408435] systemd[1]: Started Nix Daemon. gitlab # [ 219.471902] nix-daemon[2886]: accepted connection from pid 2881, user nixbot gitlab # [ 224.056738] nix-daemon[2886]: accepted connection from pid 2928, user nixbot gitlab # [ 224.077592] nix-daemon[2886]: accepted connection from pid 2881, user nixbot gitlab # [ 228.684976] nix-daemon[2886]: accepted connection from pid 2881, user nixbot gitlab # [ 228.794669] nix-daemon[2886]: accepted connection from pid 2977, user nixbot gitlab # [ 228.848582] gitaly[931]: time="2026-09-10T19:33:08.604Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CYGSYKWCRYJAWAXMNVTH7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:33:38.302" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:08.602" grpc.time_ms=0.806 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 229.009884] gitaly[931]: time="2026-09-10T19:33:08.766Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M26CYGSYKWCRYJAWAXMNVTH7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-10T19:33:18.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:33:08.759" grpc.time_ms=6.804 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 229.102713] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:08 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/b2555a8ee53ad96300d2a6b920912389a68d95c2 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 314 gitlab # [ 229.114666] nixbot[2723]: {"ts": "2026-09-10T19:33:08.859Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/b2555a8ee53ad96300d2a6b920912389a68d95c2 \"HTTP/1.1 201 Created\""} gitlab # [ 229.157085] gitaly[931]: time="2026-09-10T19:33:08.913Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CYH3YY45GTYF1877FSW9H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:33:38.611" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:08.912" grpc.time_ms=0.985 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 229.190365] gitaly[931]: time="2026-09-10T19:33:08.946Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M26CYH3YY45GTYF1877FSW9H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-10T19:33:18.190" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:33:08.940" grpc.time_ms=6.16 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 229.230939] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:08 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/b2555a8ee53ad96300d2a6b920912389a68d95c2 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 123 gitlab # [ 229.235938] nixbot[2723]: {"ts": "2026-09-10T19:33:08.992Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/b2555a8ee53ad96300d2a6b920912389a68d95c2 \"HTTP/1.1 201 Created\""} gitlab # [ 233.288937] nix-daemon[2886]: accepted connection from pid 3038, user nixbot gitlab # [ 233.369227] gitaly[931]: time="2026-09-10T19:33:13.125Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CYN7P6X4344K80QKPVXQ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:33:43.023" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:13.124" grpc.time_ms=0.572 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 233.402157] gitaly[931]: time="2026-09-10T19:33:13.158Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CYN7P6X4344K80QKPVXQ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-10T19:33:23.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:33:13.151" grpc.time_ms=7.141 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 233.470538] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:13 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/b2555a8ee53ad96300d2a6b920912389a68d95c2 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 147 gitlab # [ 233.475273] nixbot[2723]: {"ts": "2026-09-10T19:33:13.227Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/b2555a8ee53ad96300d2a6b920912389a68d95c2 \"HTTP/1.1 201 Created\""} gitlab # [ 233.750894] gitaly[931]: time="2026-09-10T19:33:13.507Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CYNKGQ4T5R4NVNGZ60RFM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:13.497" grpc.time_ms=9.535 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 233.753439] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:13 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 52 gitlab # [ 233.877517] gitaly[931]: time="2026-09-10T19:33:13.633Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M26CYNPHK5Q9VD4KRKVDKBGR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:13.620" pid=931 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 233.880211] gitaly[931]: time="2026-09-10T19:33:13.634Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M26CYNPHK5Q9VD4KRKVDKBGR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:13.620" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 233.935538] gitaly[931]: time="2026-09-10T19:33:13.691Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M26CYNPHK5Q9VD4KRKVDKBGR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:13.620" grpc.time_ms=71.254 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 233.937519] gitlab-workhorse[830]: time="2026-09-10T19:33:13Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 233.938283] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:13 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 140 gitlab # [ 234.031764] gitaly[931]: time="2026-09-10T19:33:13.788Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CYNWB3ZSZMDS6VSGCCMX0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:13.781" grpc.time_ms=6.605 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 234.032950] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:13 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 48 gitlab # [ 234.109715] gitaly[931]: time="2026-09-10T19:33:13.865Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M26CYNYNB7ATRB4SCPW2BKQ5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:13.856" pid=931 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc gitlab # [ 234.110854] gitaly[931]: time="2026-09-10T19:33:13.865Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CYNYNB7ATRB4SCPW2BKQ5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:13.856" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 234.111930] gitaly[931]: time="2026-09-10T19:33:13.866Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CYNYNB7ATRB4SCPW2BKQ5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:13.856" grpc.time_ms=9.833 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 234.168920] gitlab-workhorse[830]: time="2026-09-10T19:33:13Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 234.170043] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:13 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 111 gitlab # [ 235.591865] sidekiq-cluster[2598]: 2026-09-10T19:33:15.347Z pid=2598 tid=xam INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2598, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>449478656} gitlab # [ 235.773605] sidekiq-cluster[2598]: W, [2026-09-10T19:33:15.529878 #2598] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 235.814210] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:15.570Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 237.753911] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:17.509Z","message":"GitLab reliable fetch activated!"} gitlab # [ 237.755600] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:17.509Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 259.243853] postgres[3302]: [3302] ERROR: relation "table_without_models" does not exist gitlab # [ 259.249587] postgres[3302]: [3302] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 261.905984] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:41.659Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 261.918245] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:41.660Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 261.926696] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:41.660Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 261.934253] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:41.660Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 261.944897] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:41.665Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 44.73 seconds) gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master gitlab # [ 262.687631] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:42 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 40 gitlab # [ 262.765664] gitaly[931]: time="2026-09-10T19:33:42.521Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=7 command.system_time_ms=2 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M26CZHXD1W3PMQYRWHR9M1EM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:42.508" grpc.time_ms=12.409 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 262.809587] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:42 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.55.0" 77 gitlab # Enumerating objects: 5, done. gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. gitlab # Delta compression using up to 4 threads gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 68.00 KiB/s, done. gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) gitlab # [ 263.079532] gitaly[931]: time="2026-09-10T19:33:42.835Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=331 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CZJ295MAHNN9NZ2RG7S4B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-10T19:34:12.520" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-10T19:33:42.822" grpc.time_ms=12.893 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 263.135557] gitaly[931]: time="2026-09-10T19:33:42.891Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M26CZJ295MAHNN9NZ2RG7S4B duration_ms=142ns method=POST pid=931 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 263.173492] gitaly[931]: time="2026-09-10T19:33:42.929Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M26CZJ295MAHNN9NZ2RG7S4B duration_ms=37ns method=POST pid=931 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 263.176112] gitaly[931]: time="2026-09-10T19:33:42.930Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M26CZJ295MAHNN9NZ2RG7S4B grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-10T19:33:42.748" grpc.time_ms=181.872 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 263.241362] gitaly[931]: time="2026-09-10T19:33:42.997Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M26CZJ295MAHNN9NZ2RG7S4B grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-10T19:33:42.997" grpc.time_ms=0.441 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 263.303063] gitaly[931]: time="2026-09-10T19:33:43.059Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M26CZJ295MAHNN9NZ2RG7S4B grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-10T19:33:43.058" grpc.time_ms=0.37 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 263.324329] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:43.080Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"e2934be733b7f2d962f7550e","created_at":"2026-09-10T19:32:45.982Z","trace_propagation_headers":{"sentry-trace":"726ceca48ae041eaaedb8b76cd8277b7-236a02ab1a584453","baggage":"sentry-trace_id=726ceca48ae041eaaedb8b76cd8277b7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CXSXMCMZSAKRK9V7RXHGR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:46.009Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-e2934be733b7f2d962f7550e: start","job_status":"start","queue_duration_s":57.070514,"scheduling_latency_s":57.070585} gitlab # [ 263.367748] gitaly[931]: time="2026-09-10T19:33:43.124Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M26CZJ295MAHNN9NZ2RG7S4B grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-10T19:33:43.123" grpc.time_ms=0.376 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 263.446084] gitaly[931]: time="2026-09-10T19:33:43.202Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M26CZJ295MAHNN9NZ2RG7S4B duration_ms=39ns method=POST pid=931 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 263.447760] gitaly[931]: time="2026-09-10T19:33:43.204Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M26CZJ295MAHNN9NZ2RG7S4B grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-10T19:33:43.162" grpc.time_ms=41.543 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 263.468264] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:43.224Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"e2934be733b7f2d962f7550e","created_at":"2026-09-10T19:32:45.982Z","trace_propagation_headers":{"sentry-trace":"726ceca48ae041eaaedb8b76cd8277b7-236a02ab1a584453","baggage":"sentry-trace_id=726ceca48ae041eaaedb8b76cd8277b7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CXSXMCMZSAKRK9V7RXHGR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:46.009Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-e2934be733b7f2d962f7550e: done: 0.144542 sec","job_status":"done","queue_duration_s":57.070514,"scheduling_latency_s":57.070585,"redis_calls":24,"redis_duration_s":0.005452,"redis_read_bytes":1038,"redis_write_bytes":2604,"redis_feature_flag_calls":14,"redis_feature_flag_duration_s":0.00295,"redis_feature_flag_read_bytes":1030,"redis_feature_flag_write_bytes":1439,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00141,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001092,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.07069,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.077173217,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.144542,"completed_at":"2026-09-10T19:33:43.224Z","load_balancing_strategy":"primary","db_duration_s":0.039791,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 263.542065] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:43.226Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"7c7d6df22633231d6524ef24","created_at":"2026-09-10T19:32:46.017Z","trace_propagation_headers":{"sentry-trace":"726ceca48ae041eaaedb8b76cd8277b7-236a02ab1a584453","baggage":"sentry-trace_id=726ceca48ae041eaaedb8b76cd8277b7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CXSXMCMZSAKRK9V7RXHGR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:46.023Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-7c7d6df22633231d6524ef24: start","job_status":"start","queue_duration_s":57.202008,"scheduling_latency_s":57.202033} gitlab # [ 263.544825] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:43.301Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"7c7d6df22633231d6524ef24","created_at":"2026-09-10T19:32:46.017Z","trace_propagation_headers":{"sentry-trace":"726ceca48ae041eaaedb8b76cd8277b7-236a02ab1a584453","baggage":"sentry-trace_id=726ceca48ae041eaaedb8b76cd8277b7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CXSXMCMZSAKRK9V7RXHGR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:46.023Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-7c7d6df22633231d6524ef24: done: 0.075097 sec","job_status":"done","queue_duration_s":57.202008,"scheduling_latency_s":57.202033,"redis_calls":14,"redis_duration_s":0.003035,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000583,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001504,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000948,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055604,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.055073199999999996,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.075097,"completed_at":"2026-09-10T19:33:43.301Z","load_balancing_strategy":"primary","db_duration_s":0.013135,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 263.622366] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:43.302Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"c613a4d6bfcd6c97dd4241bc","created_at":"2026-09-10T19:32:46.574Z","trace_propagation_headers":{"sentry-trace":"726ceca48ae041eaaedb8b76cd8277b7-236a02ab1a584453","baggage":"sentry-trace_id=726ceca48ae041eaaedb8b76cd8277b7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CXSXMCMZSAKRK9V7RXHGR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:46.581Z","job_size_bytes":5,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-c613a4d6bfcd6c97dd4241bc: start","job_status":"start","queue_duration_s":56.721528,"scheduling_latency_s":56.72155} gitlab # [ 263.665929] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:43.420Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"c613a4d6bfcd6c97dd4241bc","created_at":"2026-09-10T19:32:46.574Z","trace_propagation_headers":{"sentry-trace":"726ceca48ae041eaaedb8b76cd8277b7-236a02ab1a584453","baggage":"sentry-trace_id=726ceca48ae041eaaedb8b76cd8277b7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CXSXMCMZSAKRK9V7RXHGR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:46.581Z","job_size_bytes":5,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-c613a4d6bfcd6c97dd4241bc: done: 0.118102 sec","job_status":"done","queue_duration_s":56.721528,"scheduling_latency_s":56.72155,"redis_calls":12,"redis_duration_s":0.0027,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000625,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001679,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000396,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.096458,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000371277,"duration_s":0.118102,"completed_at":"2026-09-10T19:33:43.420Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014837,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 263.667886] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:43.422Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"21efd015fb40ab1ba7729368","created_at":"2026-09-10T19:32:46.609Z","trace_propagation_headers":{"sentry-trace":"726ceca48ae041eaaedb8b76cd8277b7-236a02ab1a584453","baggage":"sentry-trace_id=726ceca48ae041eaaedb8b76cd8277b7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CXSXMCMZSAKRK9V7RXHGR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:46.615Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-21efd015fb40ab1ba7729368: start","job_status":"start","queue_duration_s":56.807036,"scheduling_latency_s":56.807059} gitlab # [ 263.763934] gitaly[931]: time="2026-09-10T19:33:43.520Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M26CZJ295MAHNN9NZ2RG7S4B diskcache=3a8c8362-3b80-4b58-8ef4-6bbd75676381 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:42.682" pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # To http://gitlab/root/test-flake.git gitlab # [ 263.780439] gitaly[931]: time="2026-09-10T19:33:43.520Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=210 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=892928 command.minflt=13065 command.oublock=32 command.real_time_ms=737 command.spawn_token_fork_ms=0 command.system_time_ms=148 command.user_time_ms=62 component=gitaly.StreamServerInterceptor correlation_id=01M26CZJ295MAHNN9NZ2RG7S4B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:42.682" grpc.time_ms=838.149 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # b2555a8..3cbe2f7 master -> master[ 263.782203] gitlab-workhorse[830]: time="2026-09-10T19:33:43Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # gitlab # [ 263.784086] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:43 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 919 gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.22 seconds) gitlab: must succeed: git -C /tmp/test-flake rev-parse master gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3cbe2f74443ff0f3ea476d1bcddd263441bc5445' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3cbe2f74443ff0f3ea476d1bcddd263441bc5445', in 0.06 seconds) gitlab # [ 263.838290] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:43.594Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"21efd015fb40ab1ba7729368","created_at":"2026-09-10T19:32:46.609Z","trace_propagation_headers":{"sentry-trace":"726ceca48ae041eaaedb8b76cd8277b7-236a02ab1a584453","baggage":"sentry-trace_id=726ceca48ae041eaaedb8b76cd8277b7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CXSXMCMZSAKRK9V7RXHGR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:46.615Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-21efd015fb40ab1ba7729368: done: 0.172121 sec","job_status":"done","queue_duration_s":56.807036,"scheduling_latency_s":56.807059,"redis_calls":11,"redis_duration_s":0.00238,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000622,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001334,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000424,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.046,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.143,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.143,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.045,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.098009,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000375746,"duration_s":0.172121,"completed_at":"2026-09-10T19:33:43.594Z","load_balancing_strategy":"primary","db_duration_s":0.074563,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 263.878197] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:43.633Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"13b985d90142c98a456b1d20","created_at":"2026-09-10T19:32:46.617Z","trace_propagation_headers":{"sentry-trace":"726ceca48ae041eaaedb8b76cd8277b7-236a02ab1a584453","baggage":"sentry-trace_id=726ceca48ae041eaaedb8b76cd8277b7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CXSXMCMZSAKRK9V7RXHGR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:46.622Z","job_size_bytes":89,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-13b985d90142c98a456b1d20: start","job_status":"start","queue_duration_s":57.011073,"scheduling_latency_s":57.011104} gitlab # [ 263.880300] nixbot[2723]: INFO: ::ffff:127.0.0.1:36992 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3cbe2f74443ff0f3ea476d1bcddd263441bc5445 HTTP/1.1" 200 OK gitlab # [ 263.908707] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:43.664Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"13b985d90142c98a456b1d20","created_at":"2026-09-10T19:32:46.617Z","trace_propagation_headers":{"sentry-trace":"726ceca48ae041eaaedb8b76cd8277b7-236a02ab1a584453","baggage":"sentry-trace_id=726ceca48ae041eaaedb8b76cd8277b7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CXSXMCMZSAKRK9V7RXHGR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:46.622Z","job_size_bytes":89,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-13b985d90142c98a456b1d20: done: 0.031222 sec","job_status":"done","queue_duration_s":57.011073,"scheduling_latency_s":57.011104,"redis_calls":11,"redis_duration_s":0.002527,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000615,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001459,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000453,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020528,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000533308,"duration_s":0.031222,"completed_at":"2026-09-10T19:33:43.664Z","load_balancing_strategy":"primary","db_duration_s":0.003569,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 263.985296] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:43.666Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 b2555a8ee53ad96300d2a6b920912389a68d95c2 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"f79b0e7c1f4b1e323c75d860","created_at":"2026-09-10T19:32:53.388Z","trace_propagation_headers":{"sentry-trace":"aa851687784a4572aae0ee315aa9bbd2-bfdc90d1a60f4a9a","baggage":"sentry-trace_id=aa851687784a4572aae0ee315aa9bbd2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CY14TA2KV1CF07DG86K3P","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:53.409Z","job_size_bytes":185,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-f79b0e7c1f4b1e323c75d860: start","job_status":"start","queue_duration_s":50.256518,"scheduling_latency_s":50.256538} gitlab # [ 263.986411] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:43.680Z","message":"changes: \"0000000000000000000000000000000000000000 b2555a8ee53ad96300d2a6b920912389a68d95c2 refs/heads/master\\n\""} gitlab # [ 263.987296] gitaly[931]: time="2026-09-10T19:33:43.714Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-10T19:33:53.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:33:43.708" grpc.time_ms=5.71 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 263.988207] gitaly[931]: time="2026-09-10T19:33:43.724Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-10T19:33:53.018" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:33:43.718" grpc.time_ms=5.29 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.060426] gitaly[931]: time="2026-09-10T19:33:43.765Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-10T19:33:53.057" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:33:43.758" grpc.time_ms=7.24 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.062420] gitaly[931]: time="2026-09-10T19:33:43.796Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3933 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-10T19:33:43.795" grpc.time_ms=0.328 pid=931 span.kind=server system=grpc gitlab # [ 264.063317] gitaly[931]: time="2026-09-10T19:33:43.817Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3935 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-10T19:33:43.817" grpc.time_ms=0.434 pid=931 span.kind=server system=grpc gitlab # [ 264.070663] gitaly[931]: time="2026-09-10T19:33:43.827Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P diskcache=b3c53174-e2d1-406e-b29b-31d80c6de842 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-10T19:33:53.073" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:33:43.774" pid=931 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 264.073353] gitaly[931]: time="2026-09-10T19:33:43.827Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=50 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4983 command.oublock=8 command.real_time_ms=46 command.spawn_token_fork_ms=0 command.system_time_ms=34 command.user_time_ms=16 component=gitaly.UnaryServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-10T19:33:53.073" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:33:43.774" grpc.time_ms=53.179 pid=931 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 264.094069] gitaly[931]: time="2026-09-10T19:33:43.845Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-10T19:33:53.144" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:33:43.845" grpc.time_ms=0.142 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.111154] gitaly[931]: time="2026-09-10T19:33:43.859Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=940 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-10T19:33:53.148" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:33:43.849" grpc.time_ms=10.127 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.161333] gitaly[931]: time="2026-09-10T19:33:43.917Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=338 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-09-10T19:34:13.012" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:43.913" grpc.time_ms=4.283 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.180640] gitaly[931]: time="2026-09-10T19:33:43.937Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-10T19:33:53.236" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:33:43.936" grpc.time_ms=0.219 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.186236] gitaly[931]: time="2026-09-10T19:33:43.942Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=344 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-10T19:33:53.238" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-09-10T19:33:43.938" grpc.time_ms=3.861 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.205578] gitaly[931]: time="2026-09-10T19:33:43.962Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-10T19:33:53.255" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:33:43.956" grpc.time_ms=5.833 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.215106] gitaly[931]: time="2026-09-10T19:33:43.971Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-10T19:33:53.266" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:33:43.966" grpc.time_ms=4.883 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.288571] gitaly[931]: time="2026-09-10T19:33:43.989Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=966 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-10T19:34:13.079" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:43.979" grpc.time_ms=9.119 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.595525] gitaly[931]: time="2026-09-10T19:33:44.351Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-10T19:33:54.009" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:44.350" grpc.time_ms=1.015 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.646398] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:44.378Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 b2555a8ee53ad96300d2a6b920912389a68d95c2 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"f79b0e7c1f4b1e323c75d860","created_at":"2026-09-10T19:32:53.388Z","trace_propagation_headers":{"sentry-trace":"aa851687784a4572aae0ee315aa9bbd2-bfdc90d1a60f4a9a","baggage":"sentry-trace_id=aa851687784a4572aae0ee315aa9bbd2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CY14TA2KV1CF07DG86K3P","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:32:53.409Z","job_size_bytes":185,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-f79b0e7c1f4b1e323c75d860: done: 0.712483 sec","job_status":"done","queue_duration_s":50.256518,"scheduling_latency_s":50.256538,"gitaly_calls":13,"gitaly_duration_s":0.152273,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.044069,"redis_read_bytes":2103,"redis_write_bytes":21749,"redis_cache_calls":15,"redis_cache_duration_s":0.003333,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.007527,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001491,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7837,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003037,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.015882,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.012799,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.033,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.198,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.214,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.033,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.477948,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000502019,"exclusive_lock_hold_duration_s":0.02661679,"duration_s":0.712483,"completed_at":"2026-09-10T19:33:44.378Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.057454,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 264.727119] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:44.381Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"98dcc3689fcd600b504cf415","created_at":"2026-09-10T19:32:58.786Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-a43624be6fc04f5a","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:58.792Z","job_size_bytes":65,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-98dcc3689fcd600b504cf415: start","job_status":"start","queue_duration_s":45.588958,"scheduling_latency_s":45.588982} gitlab # [ 264.728361] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:44.454Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"98dcc3689fcd600b504cf415","created_at":"2026-09-10T19:32:58.786Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-a43624be6fc04f5a","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:58.792Z","job_size_bytes":65,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-98dcc3689fcd600b504cf415: done: 0.073569 sec","job_status":"done","queue_duration_s":45.588958,"scheduling_latency_s":45.588982,"redis_calls":16,"redis_duration_s":0.004643,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002259,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001783,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000601,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040651,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000544483,"duration_s":0.073569,"completed_at":"2026-09-10T19:33:44.454Z","load_balancing_strategy":"primary","db_duration_s":0.024567,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 264.760829] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:44.456Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"a9ac20a8b62481f1c68c9c64","created_at":"2026-09-10T19:32:58.792Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-a43624be6fc04f5a","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:58.796Z","job_size_bytes":65,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-a9ac20a8b62481f1c68c9c64: start","job_status":"start","queue_duration_s":45.659728,"scheduling_latency_s":45.659751} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3cbe2f74443ff0f3ea476d1bcddd263441bc5445' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3cbe2f74443ff0f3ea476d1bcddd263441bc5445', in 0.02 seconds) gitlab # [ 264.845960] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:44.602Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"a9ac20a8b62481f1c68c9c64","created_at":"2026-09-10T19:32:58.792Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-a43624be6fc04f5a","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:58.796Z","job_size_bytes":65,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-a9ac20a8b62481f1c68c9c64: done: 0.145453 sec","job_status":"done","queue_duration_s":45.659728,"scheduling_latency_s":45.659751,"redis_calls":21,"redis_duration_s":0.00838,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.003673,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002409,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002298,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.106049,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000519339,"duration_s":0.145453,"completed_at":"2026-09-10T19:33:44.602Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024679,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 264.858350] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:44.603Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"0769d66a1f09d0925614482e","created_at":"2026-09-10T19:32:58.876Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-a43624be6fc04f5a","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:58.881Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-0769d66a1f09d0925614482e: start","job_status":"start","queue_duration_s":45.722403,"scheduling_latency_s":45.722459} gitlab # [ 264.933646] nixbot[2723]: INFO: ::ffff:127.0.0.1:36994 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3cbe2f74443ff0f3ea476d1bcddd263441bc5445 HTTP/1.1" 200 OK gitlab # [ 264.965505] gitaly[931]: time="2026-09-10T19:33:44.721Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M26CY6YMAKT9N9YGBJMVJJXH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-10T19:33:44.721" grpc.time_ms=0.31 pid=931 span.kind=server system=grpc gitlab # [ 264.998869] gitaly[931]: time="2026-09-10T19:33:44.755Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M26CY6YMAKT9N9YGBJMVJJXH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-10T19:33:44.755" grpc.time_ms=0.299 pid=931 span.kind=server system=grpc gitlab # [ 265.032553] gitaly[931]: time="2026-09-10T19:33:44.789Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M26CY6YMAKT9N9YGBJMVJJXH grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-10T19:33:44.788" grpc.time_ms=0.448 pid=931 span.kind=server system=grpc gitlab # [ 265.052871] gitaly[931]: time="2026-09-10T19:33:44.809Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M26CY6YMAKT9N9YGBJMVJJXH diskcache=829eb2da-c281-407a-900c-7b5da8e63e79 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-10T19:34:14.099" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:33:44.700" pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 265.053937] gitaly[931]: time="2026-09-10T19:33:44.809Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=76 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7086 command.oublock=0 command.real_time_ms=104 command.spawn_token_fork_ms=0 command.system_time_ms=55 command.user_time_ms=21 component=gitaly.UnaryServerInterceptor correlation_id=01M26CY6YMAKT9N9YGBJMVJJXH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-10T19:34:14.099" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:33:44.700" grpc.time_ms=109.334 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 265.146797] gitaly[931]: time="2026-09-10T19:33:44.903Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CY6YMAKT9N9YGBJMVJJXH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-10T19:34:14.001" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:44.902" grpc.time_ms=0.794 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 265.179245] gitaly[931]: time="2026-09-10T19:33:44.935Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CY6YMAKT9N9YGBJMVJJXH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:34:14.033" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:44.934" grpc.time_ms=1.03 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 265.189850] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:44.946Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"0769d66a1f09d0925614482e","created_at":"2026-09-10T19:32:58.876Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-a43624be6fc04f5a","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:58.881Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2598,"job_status":"rescheduled","message":"PipelineProcessWorker JID-0769d66a1f09d0925614482e: rescheduled"} gitlab # [ 265.198714] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:44.954Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"0769d66a1f09d0925614482e","created_at":"2026-09-10T19:32:58.876Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-a43624be6fc04f5a","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-10T19:32:58.881Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-0769d66a1f09d0925614482e: done: 0.351299 sec","job_status":"done","queue_duration_s":45.722403,"scheduling_latency_s":45.722459,"gitaly_calls":3,"gitaly_duration_s":0.118401,"redis_calls":120,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.029432,"redis_read_bytes":460,"redis_write_bytes":29293,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001037,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001359,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.010341,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":27,"redis_queues_duration_s":0.005227,"redis_queues_read_bytes":37,"redis_queues_write_bytes":15580,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.006744,"redis_queues_metadata_read_bytes":34,"redis_queues_metadata_write_bytes":3577,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000522,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000593,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002572,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.044,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.014,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.017,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.019,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.16332,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.303641664,"duration_s":0.351299,"completed_at":"2026-09-10T19:33:44.954Z","load_balancing_strategy":"primary","db_duration_s":0.052664,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 265.287421] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:44.957Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","b2555a8ee53ad96300d2a6b920912389a68d95c2 3cbe2f74443ff0f3ea476d1bcddd263441bc5445 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"d68bbd93f01e4db2bff11a72","created_at":"2026-09-10T19:33:43.185Z","trace_propagation_headers":{"sentry-trace":"c1e1013884fe43acbf67cc1d4c633e88-eeae8f79a7d3465a","baggage":"sentry-trace_id=c1e1013884fe43acbf67cc1d4c633e88,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CZJ295MAHNN9NZ2RG7S4B","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:43.187Z","job_size_bytes":185,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-d68bbd93f01e4db2bff11a72: start","job_status":"start","queue_duration_s":1.770078,"scheduling_latency_s":1.7701} gitlab # [ 265.288823] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:44.969Z","message":"changes: \"b2555a8ee53ad96300d2a6b920912389a68d95c2 3cbe2f74443ff0f3ea476d1bcddd263441bc5445 refs/heads/master\\n\""} gitlab # [ 265.289940] gitaly[931]: time="2026-09-10T19:33:44.993Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CZJ295MAHNN9NZ2RG7S4B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-10T19:33:54.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:33:44.987" grpc.time_ms=5.863 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 265.369677] gitaly[931]: time="2026-09-10T19:33:45.125Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZJ295MAHNN9NZ2RG7S4B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-10T19:33:55.006" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:33:45.116" grpc.time_ms=9.241 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 265.397595] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:45.153Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"7f66d641d56993b9a23d81ce","created_at":"2026-09-10T19:33:45.150Z","trace_propagation_headers":{"sentry-trace":"c1e1013884fe43acbf67cc1d4c633e88-a973c6defaf34054","baggage":"sentry-trace_id=c1e1013884fe43acbf67cc1d4c633e88,sentry-release=Unknown"},"correlation_id":"01M26CZJ295MAHNN9NZ2RG7S4B","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"e452ba95c8db12f85cf7b512","job_size_bytes":46,"pid":2598,"job_status":"deduplicated","message":"ProjectCacheWorker JID-7f66d641d56993b9a23d81ce: deduplicated: until executing","deduplication.type":"until executing"} gitlab # [ 265.433699] gitaly[931]: time="2026-09-10T19:33:45.189Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M26CZJ295MAHNN9NZ2RG7S4B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-10T19:33:55.065" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:33:45.176" grpc.time_ms=13.255 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 265.474781] gitaly[931]: time="2026-09-10T19:33:45.231Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=651 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M26CZJ295MAHNN9NZ2RG7S4B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-09-10T19:34:15.015" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:45.215" grpc.time_ms=15.139 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 265.508446] gitaly[931]: time="2026-09-10T19:33:45.264Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=343 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CZJ295MAHNN9NZ2RG7S4B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-10T19:33:55.146" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-09-10T19:33:45.256" grpc.time_ms=7.647 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 265.533923] gitaly[931]: time="2026-09-10T19:33:45.290Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CZJ295MAHNN9NZ2RG7S4B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-10T19:33:55.167" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:33:45.277" grpc.time_ms=12.388 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 265.626243] gitaly[931]: time="2026-09-10T19:33:45.382Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CZJ295MAHNN9NZ2RG7S4B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-10T19:33:55.270" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:45.380" grpc.time_ms=1.458 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 265.688716] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:45.443Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","b2555a8ee53ad96300d2a6b920912389a68d95c2 3cbe2f74443ff0f3ea476d1bcddd263441bc5445 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"d68bbd93f01e4db2bff11a72","created_at":"2026-09-10T19:33:43.185Z","trace_propagation_headers":{"sentry-trace":"c1e1013884fe43acbf67cc1d4c633e88-eeae8f79a7d3465a","baggage":"sentry-trace_id=c1e1013884fe43acbf67cc1d4c633e88,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CZJ295MAHNN9NZ2RG7S4B","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:43.187Z","job_size_bytes":185,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-d68bbd93f01e4db2bff11a72: done: 0.486181 sec","job_status":"done","queue_duration_s":1.770078,"scheduling_latency_s":1.7701,"gitaly_calls":7,"gitaly_duration_s":0.263594,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.05838,"redis_read_bytes":501,"redis_write_bytes":11942,"redis_cache_calls":9,"redis_cache_duration_s":0.003725,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.001269,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5719,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.004446,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.034281,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.014659,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.029,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.029,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.031,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.150427,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000724673,"exclusive_lock_hold_duration_s":0.134710137,"duration_s":0.486181,"completed_at":"2026-09-10T19:33:45.443Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.032949,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 265.704228] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:45.447Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"e452ba95c8db12f85cf7b512","created_at":"2026-09-10T19:33:43.726Z","trace_propagation_headers":{"sentry-trace":"aa851687784a4572aae0ee315aa9bbd2-4dc0ace573f74786","baggage":"sentry-trace_id=aa851687784a4572aae0ee315aa9bbd2,sentry-release=Unknown"},"correlation_id":"01M26CY14TA2KV1CF07DG86K3P","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:43.740Z","job_size_bytes":46,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-e452ba95c8db12f85cf7b512: start","job_status":"start","queue_duration_s":1.706507,"scheduling_latency_s":1.706559} gitlab # [ 265.789336] gitaly[931]: time="2026-09-10T19:33:45.508Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-09-11T01:33:45.504" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:33:45.505" grpc.time_ms=2.405 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 265.790877] gitaly[931]: time="2026-09-10T19:33:45.522Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=657 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-10T19:34:15.313" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:45.514" grpc.time_ms=7.694 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3cbe2f74443ff0f3ea476d1bcddd263441bc5445' gitlab # [ 265.791748] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:45.547Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"e452ba95c8db12f85cf7b512","created_at":"2026-09-10T19:33:43.726Z","trace_propagation_headers":{"sentry-trace":"aa851687784a4572aae0ee315aa9bbd2-4dc0ace573f74786","baggage":"sentry-trace_id=aa851687784a4572aae0ee315aa9bbd2,sentry-release=Unknown"},"correlation_id":"01M26CY14TA2KV1CF07DG86K3P","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:43.740Z","job_size_bytes":46,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-e452ba95c8db12f85cf7b512: done: 0.100932 sec","job_status":"done","queue_duration_s":1.706507,"scheduling_latency_s":1.706559,"gitaly_calls":2,"gitaly_duration_s":0.014983,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.013651,"redis_read_bytes":69,"redis_write_bytes":4887,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.002805,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000903,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2115,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003209,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.00501,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001724,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053626,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000625219,"duration_s":0.100932,"completed_at":"2026-09-10T19:33:45.547Z","load_balancing_strategy":"primary","db_duration_s":0.015625,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 265.905667] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:45.549Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"ab6d9e873f2595f469b0e01c","created_at":"2026-09-10T19:33:43.750Z","trace_propagation_headers":{"sentry-trace":"aa851687784a4572aae0ee315aa9bbd2-4dc0ace573f74786","baggage":"sentry-trace_id=aa851687784a4572aae0ee315aa9bbd2,sentry-release=Unknown"},"correlation_id":"01M26CY14TA2KV1CF07DG86K3P","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:43.754Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-ab6d9e873f2595f469b0e01c: start","job_status":"start","queue_duration_s":1.795497,"scheduling_latency_s":1.795519} gitlab # [ 265.906912] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:45.633Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"ab6d9e873f2595f469b0e01c","created_at":"2026-09-10T19:33:43.750Z","trace_propagation_headers":{"sentry-trace":"aa851687784a4572aae0ee315aa9bbd2-4dc0ace573f74786","baggage":"sentry-trace_id=aa851687784a4572aae0ee315aa9bbd2,sentry-release=Unknown"},"correlation_id":"01M26CY14TA2KV1CF07DG86K3P","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:43.754Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-ab6d9e873f2595f469b0e01c: done: 0.083157 sec","job_status":"done","queue_duration_s":1.795497,"scheduling_latency_s":1.795519,"gitaly_calls":1,"gitaly_duration_s":0.027438,"redis_calls":15,"redis_duration_s":0.005248,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000765,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001811,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001159,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001513,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.01,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.017,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035366,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.059605899,"duration_s":0.083157,"completed_at":"2026-09-10T19:33:45.632Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014323,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.942956] gitaly[931]: time="2026-09-10T19:33:45.599Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P diskcache=e453b080-4eca-4aea-a666-9e243647e37f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-11T01:33:45.574" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:45.576" pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 265.958382] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:45.634Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"f7116b5d770edf1b0e710087","created_at":"2026-09-10T19:33:43.840Z","trace_propagation_headers":{"sentry-trace":"aa851687784a4572aae0ee315aa9bbd2-4dc0ace573f74786","baggage":"sentry-trace_id=aa851687784a4572aae0ee315aa9bbd2,sentry-release=Unknown"},"correlation_id":"01M26CY14TA2KV1CF07DG86K3P","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:43.842Z","job_size_bytes":23,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-f7116b5d770edf1b0e710087: start","job_status":"start","queue_duration_s":1.791766,"scheduling_latency_s":1.791788} gitlab # [ 266.027264] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:45.671Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"f7116b5d770edf1b0e710087","created_at":"2026-09-10T19:33:43.840Z","trace_propagation_headers":{"sentry-trace":"aa851687784a4572aae0ee315aa9bbd2-4dc0ace573f74786","baggage":"sentry-trace_id=aa851687784a4572aae0ee315aa9bbd2,sentry-release=Unknown"},"correlation_id":"01M26CY14TA2KV1CF07DG86K3P","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:43.842Z","job_size_bytes":23,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-f7116b5d770edf1b0e710087: done: 0.036767 sec","job_status":"done","queue_duration_s":1.791766,"scheduling_latency_s":1.791788,"gitaly_calls":1,"gitaly_duration_s":0.009007,"redis_calls":16,"redis_duration_s":0.003458,"redis_read_bytes":52,"redis_write_bytes":2047,"redis_queues_calls":1,"redis_queues_duration_s":0.000239,"redis_queues_read_bytes":1,"redis_queues_write_bytes":978,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001012,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001537,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00067,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021445,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036767,"completed_at":"2026-09-10T19:33:45.671Z","load_balancing_strategy":"primary","db_duration_s":0.002278,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 266.099803] gitaly[931]: time="2026-09-10T19:33:45.599Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1277 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=4 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-11T01:33:45.574" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:45.576" grpc.time_ms=23.603 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 266.100985] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:45.674Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"06b6ed02a1801644174c0a7e","created_at":"2026-09-10T19:33:44.234Z","trace_propagation_headers":{"sentry-trace":"aa851687784a4572aae0ee315aa9bbd2-4dc0ace573f74786","baggage":"sentry-trace_id=aa851687784a4572aae0ee315aa9bbd2,sentry-release=Unknown"},"correlation_id":"01M26CY14TA2KV1CF07DG86K3P","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e28330498ca0a2876c222a0066993b7e616f1290dc4c586c444e296a91dd1e7f","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.239Z","job_size_bytes":467,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-06b6ed02a1801644174c0a7e: start","job_status":"start","queue_duration_s":1.435372,"scheduling_latency_s":1.435399} gitlab # [ 266.102344] gitaly[931]: time="2026-09-10T19:33:45.663Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=651 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-10T19:34:15.456" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:45.656" grpc.time_ms=6.913 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 266.123952] gitaly[931]: time="2026-09-10T19:33:45.697Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:34:15.496" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:45.696" grpc.time_ms=1.007 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 266.198957] gitaly[931]: time="2026-09-10T19:33:45.708Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-10T19:33:55.009" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:33:45.699" grpc.time_ms=8.491 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 266.200247] gitaly[931]: time="2026-09-10T19:33:45.788Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-10T19:34:15.586" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:45.787" grpc.time_ms=0.93 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 266.201293] gitaly[931]: time="2026-09-10T19:33:45.793Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=90 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-10T19:34:15.590" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:45.791" grpc.time_ms=2.052 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 266.208077] gitaly[931]: time="2026-09-10T19:33:45.852Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-10T19:33:55.155" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:33:45.846" grpc.time_ms=5.547 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 266.309987] nixbot[2723]: INFO: ::ffff:127.0.0.1:37008 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3cbe2f74443ff0f3ea476d1bcddd263441bc5445 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3cbe2f74443ff0f3ea476d1bcddd263441bc5445', in 0.41 seconds) gitlab # [ 266.485105] gitaly[931]: time="2026-09-10T19:33:46.241Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=969 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-10T19:34:16.028" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:46.228" grpc.time_ms=12.931 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 266.556057] gitaly[931]: time="2026-09-10T19:33:46.312Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CY14TA2KV1CF07DG86K3P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-10T19:34:16.101" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:46.301" grpc.time_ms=10.702 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 266.596058] sidekiq-cluster[2598]: {"severity":"WARN","time":"2026-09-10T19:33:46.352Z","class":"CreatePipelineWorker","correlation_id":"01M26CY14TA2KV1CF07DG86K3P","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"b2555a8ee53ad96300d2a6b920912389a68d95c2","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"b2555a8ee53ad96300d2a6b920912389a68d95c2","ref":"refs/heads/master"},"retry":0} gitlab # [ 266.606866] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:46.362Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"06b6ed02a1801644174c0a7e","created_at":"2026-09-10T19:33:44.234Z","trace_propagation_headers":{"sentry-trace":"aa851687784a4572aae0ee315aa9bbd2-4dc0ace573f74786","baggage":"sentry-trace_id=aa851687784a4572aae0ee315aa9bbd2,sentry-release=Unknown"},"correlation_id":"01M26CY14TA2KV1CF07DG86K3P","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e28330498ca0a2876c222a0066993b7e616f1290dc4c586c444e296a91dd1e7f","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.239Z","job_size_bytes":467,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-06b6ed02a1801644174c0a7e: done: 0.68808 sec","job_status":"done","queue_duration_s":1.435372,"scheduling_latency_s":1.435399,"gitaly_calls":7,"gitaly_duration_s":0.113289,"redis_calls":51,"redis_duration_s":0.010185,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.000971,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.004246,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001531,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002195,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001242,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.518112,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000674667,"duration_s":0.68808,"completed_at":"2026-09-10T19:33:46.362Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.040596,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 266.614433] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:46.366Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"ebd034f9-bbba-4976-a83c-9d6f8ec95bec\"}"],"class":"WebHookWorker","jid":"adf767f74ca5876aa3d7cb5b","created_at":"2026-09-10T19:33:44.328Z","trace_propagation_headers":{"sentry-trace":"aa851687784a4572aae0ee315aa9bbd2-4dc0ace573f74786","baggage":"sentry-trace_id=aa851687784a4572aae0ee315aa9bbd2,sentry-release=Unknown"},"correlation_id":"01M26CY14TA2KV1CF07DG86K3P","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11610364ca9fecbdc9c873c228ac620bb58b983c02b112028c011d04925405a1","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.333Z","job_size_bytes":1751,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-adf767f74ca5876aa3d7cb5b: start","job_status":"start","queue_duration_s":2.032953,"scheduling_latency_s":2.032994} gitlab # [ 266.846534] nixbot[2723]: INFO: ::ffff:127.0.0.1:37010 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 266.878169] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:46.632Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"ebd034f9-bbba-4976-a83c-9d6f8ec95bec\"}"],"class":"WebHookWorker","jid":"adf767f74ca5876aa3d7cb5b","created_at":"2026-09-10T19:33:44.328Z","trace_propagation_headers":{"sentry-trace":"aa851687784a4572aae0ee315aa9bbd2-4dc0ace573f74786","baggage":"sentry-trace_id=aa851687784a4572aae0ee315aa9bbd2,sentry-release=Unknown"},"correlation_id":"01M26CY14TA2KV1CF07DG86K3P","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11610364ca9fecbdc9c873c228ac620bb58b983c02b112028c011d04925405a1","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.333Z","job_size_bytes":1751,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-adf767f74ca5876aa3d7cb5b: done: 0.26611 sec","job_status":"done","queue_duration_s":2.032953,"scheduling_latency_s":2.032994,"redis_calls":19,"redis_duration_s":0.008022,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001866,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000761,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003248,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002147,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.088,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.088,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.06301387499996736,"cpu_s":0.093977,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000937549,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.26611,"completed_at":"2026-09-10T19:33:46.632Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.087566,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.893961] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:46.639Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"67a1d43fb6138f83760f885c","created_at":"2026-09-10T19:33:44.687Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-6ed920e1266f4b87","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.691Z","job_size_bytes":23,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-67a1d43fb6138f83760f885c: start","job_status":"start","queue_duration_s":1.948067,"scheduling_latency_s":1.948093} gitlab # [ 266.983470] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:46.708Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"67a1d43fb6138f83760f885c","created_at":"2026-09-10T19:33:44.687Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-6ed920e1266f4b87","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.691Z","job_size_bytes":23,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-67a1d43fb6138f83760f885c: done: 0.069515 sec","job_status":"done","queue_duration_s":1.948067,"scheduling_latency_s":1.948093,"redis_calls":10,"redis_duration_s":0.003157,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000754,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001754,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000649,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044766,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000350324,"duration_s":0.069515,"completed_at":"2026-09-10T19:33:46.708Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018275,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 267.011193] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:46.710Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"7e41c49f7728702c89cc075f","created_at":"2026-09-10T19:33:44.815Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-6ed920e1266f4b87","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.819Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-7e41c49f7728702c89cc075f: start","job_status":"start","queue_duration_s":1.890829,"scheduling_latency_s":1.890852} gitlab # [ 267.090197] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:46.740Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"7e41c49f7728702c89cc075f","created_at":"2026-09-10T19:33:44.815Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-6ed920e1266f4b87","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.819Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-7e41c49f7728702c89cc075f: done: 0.03019 sec","job_status":"done","queue_duration_s":1.890829,"scheduling_latency_s":1.890852,"redis_calls":10,"redis_duration_s":0.002108,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000437,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001231,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00044,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017709,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034781,"duration_s":0.03019,"completed_at":"2026-09-10T19:33:46.740Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006927,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.161063] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:46.742Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"260d0d066efe88af9801c046","created_at":"2026-09-10T19:33:44.828Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-6ed920e1266f4b87","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.833Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-260d0d066efe88af9801c046: start","job_status":"start","queue_duration_s":1.909089,"scheduling_latency_s":1.90911} gitlab # [ 267.162262] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:46.787Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"260d0d066efe88af9801c046","created_at":"2026-09-10T19:33:44.828Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-6ed920e1266f4b87","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.833Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-260d0d066efe88af9801c046: done: 0.044944 sec","job_status":"done","queue_duration_s":1.909089,"scheduling_latency_s":1.90911,"redis_calls":10,"redis_duration_s":0.003797,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000917,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002176,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000704,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027811,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000647568,"duration_s":0.044944,"completed_at":"2026-09-10T19:33:46.787Z","load_balancing_strategy":"primary","db_duration_s":0.006667,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.259406] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:46.790Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"a29b3262cfe7452d645834ee","created_at":"2026-09-10T19:33:44.842Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-6ed920e1266f4b87","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.847Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-a29b3262cfe7452d645834ee: start","job_status":"start","queue_duration_s":1.94297,"scheduling_latency_s":1.943011} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3cbe2f74443ff0f3ea476d1bcddd263441bc5445' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3cbe2f74443ff0f3ea476d1bcddd263441bc5445', in 0.02 seconds) gitlab # [ 267.260973] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:46.826Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"a29b3262cfe7452d645834ee","created_at":"2026-09-10T19:33:44.842Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-6ed920e1266f4b87","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.847Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-a29b3262cfe7452d645834ee: done: 0.036961 sec","job_status":"done","queue_duration_s":1.94297,"scheduling_latency_s":1.943011,"redis_calls":13,"redis_duration_s":0.003064,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000454,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001455,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000568,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000587,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02405,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000401727,"duration_s":0.036961,"completed_at":"2026-09-10T19:33:46.826Z","load_balancing_strategy":"primary","db_duration_s":0.004051,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.340314] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:46.828Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"616c6aede3eb2c6045225978","created_at":"2026-09-10T19:33:44.847Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-6ed920e1266f4b87","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.850Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-616c6aede3eb2c6045225978: start","job_status":"start","queue_duration_s":1.97749,"scheduling_latency_s":1.977511} gitlab # [ 267.341406] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:46.854Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"616c6aede3eb2c6045225978","created_at":"2026-09-10T19:33:44.847Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-6ed920e1266f4b87","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.850Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-616c6aede3eb2c6045225978: done: 0.025915 sec","job_status":"done","queue_duration_s":1.97749,"scheduling_latency_s":1.977511,"redis_calls":10,"redis_duration_s":0.001959,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000374,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001202,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000383,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017133,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000386083,"duration_s":0.025915,"completed_at":"2026-09-10T19:33:46.854Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003554,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.415956] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:46.855Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"854b0a6df0b192ff3fe3031a","created_at":"2026-09-10T19:33:44.851Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-6ed920e1266f4b87","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.855Z","job_size_bytes":19,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-854b0a6df0b192ff3fe3031a: start","job_status":"start","queue_duration_s":2.000517,"scheduling_latency_s":2.000537} gitlab # [ 267.417369] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:46.879Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"854b0a6df0b192ff3fe3031a","created_at":"2026-09-10T19:33:44.851Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-6ed920e1266f4b87","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.855Z","job_size_bytes":19,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-854b0a6df0b192ff3fe3031a: done: 0.023877 sec","job_status":"done","queue_duration_s":2.000517,"scheduling_latency_s":2.000537,"redis_calls":11,"redis_duration_s":0.002305,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000462,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001454,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000389,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015346,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000331047,"duration_s":0.023877,"completed_at":"2026-09-10T19:33:46.879Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002684,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.518949] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:46.880Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"1aa7b6dfb2ce20e617f3c4cb","created_at":"2026-09-10T19:33:44.857Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-6ed920e1266f4b87","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.860Z","job_size_bytes":66,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-1aa7b6dfb2ce20e617f3c4cb: start","job_status":"start","queue_duration_s":2.020707,"scheduling_latency_s":2.020727} gitlab # [ 267.520854] nixbot[2723]: INFO: ::ffff:127.0.0.1:37018 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3cbe2f74443ff0f3ea476d1bcddd263441bc5445 HTTP/1.1" 200 OK gitlab # [ 267.522217] gitaly[931]: time="2026-09-10T19:33:47.256Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M26CZJ295MAHNN9NZ2RG7S4B diskcache=4b948056-bd23-491f-80ad-b10a2695878c grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-11T01:33:47.247" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:47.247" pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 267.526133] gitaly[931]: time="2026-09-10T19:33:47.256Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=315 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZJ295MAHNN9NZ2RG7S4B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-11T01:33:47.247" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:47.247" grpc.time_ms=8.882 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 267.596269] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:46.969Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"1aa7b6dfb2ce20e617f3c4cb","created_at":"2026-09-10T19:33:44.857Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-6ed920e1266f4b87","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.860Z","job_size_bytes":66,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-1aa7b6dfb2ce20e617f3c4cb: done: 0.088926 sec","job_status":"done","queue_duration_s":2.020707,"scheduling_latency_s":2.020727,"redis_calls":10,"redis_duration_s":0.0023239999999999997,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00052,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001334,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00047,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.063,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.061,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019061,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000369041,"duration_s":0.088926,"completed_at":"2026-09-10T19:33:46.969Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.064214,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.616972] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:46.971Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"32063f8b03984bf3d2f2a23f","created_at":"2026-09-10T19:33:44.860Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-6ed920e1266f4b87","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.864Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-32063f8b03984bf3d2f2a23f: start","job_status":"start","queue_duration_s":2.106968,"scheduling_latency_s":2.106991} gitlab # [ 267.685965] gitaly[931]: time="2026-09-10T19:33:47.290Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZJ295MAHNN9NZ2RG7S4B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:34:17.089" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:47.289" grpc.time_ms=0.468 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 267.687209] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:47.002Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"32063f8b03984bf3d2f2a23f","created_at":"2026-09-10T19:33:44.860Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-6ed920e1266f4b87","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.864Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-32063f8b03984bf3d2f2a23f: done: 0.031579 sec","job_status":"done","queue_duration_s":2.106968,"scheduling_latency_s":2.106991,"redis_calls":10,"redis_duration_s":0.002017,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00038,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001248,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000389,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021929,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000334121,"duration_s":0.031579,"completed_at":"2026-09-10T19:33:47.002Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003846,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.789282] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:47.004Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"7c5cf19fdd77f8ede7cbfce8","created_at":"2026-09-10T19:33:44.871Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-6ed920e1266f4b87","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.874Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-7c5cf19fdd77f8ede7cbfce8: start","job_status":"start","queue_duration_s":2.129414,"scheduling_latency_s":2.129435} gitlab # [ 267.790398] gitaly[931]: time="2026-09-10T19:33:47.297Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZJ295MAHNN9NZ2RG7S4B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-10T19:33:57.181" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:33:47.292" grpc.time_ms=5.487 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 267.791457] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:47.035Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"7c5cf19fdd77f8ede7cbfce8","created_at":"2026-09-10T19:33:44.871Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-6ed920e1266f4b87","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.874Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-7c5cf19fdd77f8ede7cbfce8: done: 0.0313 sec","job_status":"done","queue_duration_s":2.129414,"scheduling_latency_s":2.129435,"redis_calls":10,"redis_duration_s":0.002129,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000468,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001233,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000428,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017147,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000342223,"duration_s":0.0313,"completed_at":"2026-09-10T19:33:47.035Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008988,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.871209] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:47.037Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"55a80d1daaf42327adc8f530","created_at":"2026-09-10T19:33:44.875Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-6ed920e1266f4b87","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.879Z","job_size_bytes":6,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-55a80d1daaf42327adc8f530: start","job_status":"start","queue_duration_s":2.157739,"scheduling_latency_s":2.157759} gitlab # [ 267.872778] gitaly[931]: time="2026-09-10T19:33:47.323Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CZJ295MAHNN9NZ2RG7S4B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-10T19:34:17.122" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:47.322" grpc.time_ms=0.72 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 267.953600] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:47.056Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"55a80d1daaf42327adc8f530","created_at":"2026-09-10T19:33:44.875Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-6ed920e1266f4b87","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.879Z","job_size_bytes":6,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-55a80d1daaf42327adc8f530: done: 0.019329 sec","job_status":"done","queue_duration_s":2.157739,"scheduling_latency_s":2.157759,"redis_calls":10,"redis_duration_s":0.0020340000000000002,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00038,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001227,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000427,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000339987,"duration_s":0.019329,"completed_at":"2026-09-10T19:33:47.056Z","load_balancing_strategy":"primary","db_duration_s":0.001526,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.956114] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:47.057Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"15f36b2c7484e52d973aee1b","created_at":"2026-09-10T19:33:44.879Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-6ed920e1266f4b87","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.883Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-15f36b2c7484e52d973aee1b: start","job_status":"start","queue_duration_s":2.174321,"scheduling_latency_s":2.174343} gitlab # [ 268.033962] gitaly[931]: time="2026-09-10T19:33:47.328Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=228 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CZJ295MAHNN9NZ2RG7S4B error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-10T19:34:17.125" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:47.326" grpc.time_ms=2.443 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 268.035248] nixbot[2723]: INFO: ::ffff:127.0.0.1:37032 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 268.036112] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:47.155Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"15f36b2c7484e52d973aee1b","created_at":"2026-09-10T19:33:44.879Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-6ed920e1266f4b87","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.883Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-15f36b2c7484e52d973aee1b: done: 0.098143 sec","job_status":"done","queue_duration_s":2.174321,"scheduling_latency_s":2.174343,"redis_calls":10,"redis_duration_s":0.002194,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00038,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001396,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000418,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.08459,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000406476,"duration_s":0.098143,"completed_at":"2026-09-10T19:33:47.155Z","load_balancing_strategy":"primary","db_duration_s":0.005111,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.105361] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:47.157Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"a4984569f1d03e0c22d3b0e9","created_at":"2026-09-10T19:33:44.888Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-6ed920e1266f4b87","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.892Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-a4984569f1d03e0c22d3b0e9: start","job_status":"start","queue_duration_s":2.2647,"scheduling_latency_s":2.264723} gitlab # [ 268.106931] gitaly[931]: time="2026-09-10T19:33:47.352Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M26CZJ295MAHNN9NZ2RG7S4B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-10T19:33:57.236" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:33:47.346" grpc.time_ms=6.265 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 268.119308] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:47.206Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"a4984569f1d03e0c22d3b0e9","created_at":"2026-09-10T19:33:44.888Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-6ed920e1266f4b87","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.892Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-a4984569f1d03e0c22d3b0e9: done: 0.049337 sec","job_status":"done","queue_duration_s":2.2647,"scheduling_latency_s":2.264723,"redis_calls":14,"redis_duration_s":0.003295,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000388,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001297,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00161,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032823,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034334,"duration_s":0.049337,"completed_at":"2026-09-10T19:33:47.206Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009216,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.188291] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:47.208Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"536932ca0bca5f4f516b1e7d","created_at":"2026-09-10T19:33:44.946Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-6ed920e1266f4b87","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.947Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-536932ca0bca5f4f516b1e7d: start","job_status":"start","queue_duration_s":2.260198,"scheduling_latency_s":2.26022} gitlab # [ 268.211370] gitaly[931]: time="2026-09-10T19:33:47.676Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=961 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M26CZJ295MAHNN9NZ2RG7S4B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-10T19:34:17.466" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:47.667" grpc.time_ms=9.4 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3cbe2f74443ff0f3ea476d1bcddd263441bc5445' gitlab # [ 268.235241] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:47.230Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"536932ca0bca5f4f516b1e7d","created_at":"2026-09-10T19:33:44.946Z","trace_propagation_headers":{"sentry-trace":"a96716d61162462a982d203d4e48177d-6ed920e1266f4b87","baggage":"sentry-trace_id=a96716d61162462a982d203d4e48177d,sentry-release=Unknown"},"correlation_id":"01M26CY6YMAKT9N9YGBJMVJJXH","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:44.947Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-536932ca0bca5f4f516b1e7d: done: 0.022095 sec","job_status":"done","queue_duration_s":2.260198,"scheduling_latency_s":2.26022,"redis_calls":8,"redis_duration_s":0.001598,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00122,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000378,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015189,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022095,"completed_at":"2026-09-10T19:33:47.230Z","load_balancing_strategy":"primary","db_duration_s":0.002937,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3cbe2f74443ff0f3ea476d1bcddd263441bc5445', in 0.03 seconds) gitlab # [ 268.308195] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:47.231Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"ac2510d8fc164453273eac3a","created_at":"2026-09-10T19:33:45.162Z","trace_propagation_headers":{"sentry-trace":"c1e1013884fe43acbf67cc1d4c633e88-a973c6defaf34054","baggage":"sentry-trace_id=c1e1013884fe43acbf67cc1d4c633e88,sentry-release=Unknown"},"correlation_id":"01M26CZJ295MAHNN9NZ2RG7S4B","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"ab6d9e873f2595f469b0e01c","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:45.166Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-ac2510d8fc164453273eac3a: start","job_status":"start","queue_duration_s":2.064586,"scheduling_latency_s":2.064607} gitlab # [ 268.387468] gitaly[931]: time="2026-09-10T19:33:47.686Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CZJ295MAHNN9NZ2RG7S4B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-10T19:34:17.478" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:47.678" grpc.time_ms=7.392 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 268.388945] nixbot[2723]: INFO: ::ffff:127.0.0.1:37044 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3cbe2f74443ff0f3ea476d1bcddd263441bc5445 HTTP/1.1" 200 OK gitlab # [ 268.390112] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:47.269Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"ac2510d8fc164453273eac3a","created_at":"2026-09-10T19:33:45.162Z","trace_propagation_headers":{"sentry-trace":"c1e1013884fe43acbf67cc1d4c633e88-a973c6defaf34054","baggage":"sentry-trace_id=c1e1013884fe43acbf67cc1d4c633e88,sentry-release=Unknown"},"correlation_id":"01M26CZJ295MAHNN9NZ2RG7S4B","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"ab6d9e873f2595f469b0e01c","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:45.166Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-ac2510d8fc164453273eac3a: done: 0.037463 sec","job_status":"done","queue_duration_s":2.064586,"scheduling_latency_s":2.064607,"gitaly_calls":1,"gitaly_duration_s":0.010981,"redis_calls":12,"redis_duration_s":0.002594,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001094,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000561,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000939,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019988,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.021615876,"duration_s":0.037463,"completed_at":"2026-09-10T19:33:47.268Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003822,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.418164] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:47.270Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"659190b68b444b4692f8388c","created_at":"2026-09-10T19:33:45.354Z","trace_propagation_headers":{"sentry-trace":"c1e1013884fe43acbf67cc1d4c633e88-a973c6defaf34054","baggage":"sentry-trace_id=c1e1013884fe43acbf67cc1d4c633e88,sentry-release=Unknown"},"correlation_id":"01M26CZJ295MAHNN9NZ2RG7S4B","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:927f28d26276c5d49714200b016118bfd4bd2b96e96ca89f9dc17c78a2277015","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:45.356Z","job_size_bytes":467,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-659190b68b444b4692f8388c: start","job_status":"start","queue_duration_s":1.914202,"scheduling_latency_s":1.914223} gitlab # [ 268.445117] sidekiq-cluster[2598]: {"severity":"WARN","time":"2026-09-10T19:33:47.695Z","class":"CreatePipelineWorker","correlation_id":"01M26CZJ295MAHNN9NZ2RG7S4B","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"b2555a8ee53ad96300d2a6b920912389a68d95c2","after":"3cbe2f74443ff0f3ea476d1bcddd263441bc5445","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"3cbe2f74443ff0f3ea476d1bcddd263441bc5445","ref":"refs/heads/master"},"retry":0} gitlab # [ 268.456180] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:47.699Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"659190b68b444b4692f8388c","created_at":"2026-09-10T19:33:45.354Z","trace_propagation_headers":{"sentry-trace":"c1e1013884fe43acbf67cc1d4c633e88-a973c6defaf34054","baggage":"sentry-trace_id=c1e1013884fe43acbf67cc1d4c633e88,sentry-release=Unknown"},"correlation_id":"01M26CZJ295MAHNN9NZ2RG7S4B","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:927f28d26276c5d49714200b016118bfd4bd2b96e96ca89f9dc17c78a2277015","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:45.356Z","job_size_bytes":467,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-659190b68b444b4692f8388c: done: 0.42937 sec","job_status":"done","queue_duration_s":1.914202,"scheduling_latency_s":1.914223,"gitaly_calls":7,"gitaly_duration_s":0.044509,"redis_calls":33,"redis_duration_s":0.004726,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.00058,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001061,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.00201,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001075,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.365458,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.42937,"completed_at":"2026-09-10T19:33:47.699Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012573,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 268.529504] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:47.701Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"60a643d3-abf5-4c05-b1e8-fbddcb9a2d52\"}"],"class":"WebHookWorker","jid":"4511c0369016428c24316c7d","created_at":"2026-09-10T19:33:45.368Z","trace_propagation_headers":{"sentry-trace":"c1e1013884fe43acbf67cc1d4c633e88-a973c6defaf34054","baggage":"sentry-trace_id=c1e1013884fe43acbf67cc1d4c633e88,sentry-release=Unknown"},"correlation_id":"01M26CZJ295MAHNN9NZ2RG7S4B","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ead0ac0faf388315edeadada84d25a7139b0b580d42cada13f37111500ff7b98","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:45.369Z","job_size_bytes":1749,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-4511c0369016428c24316c7d: start","job_status":"start","queue_duration_s":2.331364,"scheduling_latency_s":2.331389} gitlab # [ 268.610443] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:47.731Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"60a643d3-abf5-4c05-b1e8-fbddcb9a2d52\"}"],"class":"WebHookWorker","jid":"4511c0369016428c24316c7d","created_at":"2026-09-10T19:33:45.368Z","trace_propagation_headers":{"sentry-trace":"c1e1013884fe43acbf67cc1d4c633e88-a973c6defaf34054","baggage":"sentry-trace_id=c1e1013884fe43acbf67cc1d4c633e88,sentry-release=Unknown"},"correlation_id":"01M26CZJ295MAHNN9NZ2RG7S4B","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ead0ac0faf388315edeadada84d25a7139b0b580d42cada13f37111500ff7b98","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:45.369Z","job_size_bytes":1749,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-4511c0369016428c24316c7d: done: 0.0302 sec","job_status":"done","queue_duration_s":2.331364,"scheduling_latency_s":2.331389,"redis_calls":14,"redis_duration_s":0.002252,"redis_read_bytes":25,"redis_write_bytes":4684,"redis_queues_calls":2,"redis_queues_duration_s":0.000264,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3728,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001251,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000737,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.005186691999995219,"cpu_s":0.018709,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.0302,"completed_at":"2026-09-10T19:33:47.731Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001374,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.628490] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:47.732Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"389652407647f233260a66d6","created_at":"2026-09-10T19:33:45.531Z","trace_propagation_headers":{"sentry-trace":"aa851687784a4572aae0ee315aa9bbd2-b1267d5850084cef","baggage":"sentry-trace_id=aa851687784a4572aae0ee315aa9bbd2,sentry-release=Unknown"},"correlation_id":"01M26CY14TA2KV1CF07DG86K3P","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:45.536Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-389652407647f233260a66d6: start","job_status":"start","queue_duration_s":2.196362,"scheduling_latency_s":2.196385} gitlab # [ 268.653429] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:47.768Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"389652407647f233260a66d6","created_at":"2026-09-10T19:33:45.531Z","trace_propagation_headers":{"sentry-trace":"aa851687784a4572aae0ee315aa9bbd2-b1267d5850084cef","baggage":"sentry-trace_id=aa851687784a4572aae0ee315aa9bbd2,sentry-release=Unknown"},"correlation_id":"01M26CY14TA2KV1CF07DG86K3P","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:45.536Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-389652407647f233260a66d6: done: 0.035553 sec","job_status":"done","queue_duration_s":2.196362,"scheduling_latency_s":2.196385,"redis_calls":19,"redis_duration_s":0.003716,"redis_read_bytes":21,"redis_write_bytes":4291,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000764,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000471,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2262,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001844,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000637,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024598,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.006024788,"duration_s":0.035553,"completed_at":"2026-09-10T19:33:47.768Z","load_balancing_strategy":"primary","db_duration_s":0.005228,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.760515] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:47.770Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","adf767f74ca5876aa3d7cb5b"],"class":"WebHooks::LogExecutionWorker","jid":"1e424fa0989fd8d55fb05308","created_at":"2026-09-10T19:33:46.606Z","trace_propagation_headers":{"sentry-trace":"aa851687784a4572aae0ee315aa9bbd2-ab22b33e0e9d4ec4","baggage":"sentry-trace_id=aa851687784a4572aae0ee315aa9bbd2,sentry-release=Unknown"},"correlation_id":"01M26CY14TA2KV1CF07DG86K3P","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:40d111892c51982f5c2294e8d4888af25573ddf81645e359e7eec9f17d6e02b2","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:46.616Z","job_size_bytes":2593,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-1e424fa0989fd8d55fb05308: start","job_status":"start","queue_duration_s":1.153698,"scheduling_latency_s":1.15372} gitlab # [ 268.761961] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:47.803Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","adf767f74ca5876aa3d7cb5b"],"class":"WebHooks::LogExecutionWorker","jid":"1e424fa0989fd8d55fb05308","created_at":"2026-09-10T19:33:46.606Z","trace_propagation_headers":{"sentry-trace":"aa851687784a4572aae0ee315aa9bbd2-ab22b33e0e9d4ec4","baggage":"sentry-trace_id=aa851687784a4572aae0ee315aa9bbd2,sentry-release=Unknown"},"correlation_id":"01M26CY14TA2KV1CF07DG86K3P","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:40d111892c51982f5c2294e8d4888af25573ddf81645e359e7eec9f17d6e02b2","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:46.616Z","job_size_bytes":2593,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-1e424fa0989fd8d55fb05308: done: 0.033603 sec","job_status":"done","queue_duration_s":1.153698,"scheduling_latency_s":1.15372,"redis_calls":10,"redis_duration_s":0.00206,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000393,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001273,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000394,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022958,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000334958,"duration_s":0.033603,"completed_at":"2026-09-10T19:33:47.803Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005498,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.819439] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:47.805Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","4511c0369016428c24316c7d"],"class":"WebHooks::LogExecutionWorker","jid":"ba51e23a0c05ca29472c631f","created_at":"2026-09-10T19:33:47.725Z","trace_propagation_headers":{"sentry-trace":"c1e1013884fe43acbf67cc1d4c633e88-8152e021539c42ce","baggage":"sentry-trace_id=c1e1013884fe43acbf67cc1d4c633e88,sentry-release=Unknown"},"correlation_id":"01M26CZJ295MAHNN9NZ2RG7S4B","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:af25898a4e4380d4672aa4894e0ab2d8b944b16ae68493f0501e8456d9bc2527","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:47.726Z","job_size_bytes":2592,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-ba51e23a0c05ca29472c631f: start","job_status":"start","queue_duration_s":0.07807,"scheduling_latency_s":0.07809} gitlab # [ 268.845359] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:47.824Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","4511c0369016428c24316c7d"],"class":"WebHooks::LogExecutionWorker","jid":"ba51e23a0c05ca29472c631f","created_at":"2026-09-10T19:33:47.725Z","trace_propagation_headers":{"sentry-trace":"c1e1013884fe43acbf67cc1d4c633e88-8152e021539c42ce","baggage":"sentry-trace_id=c1e1013884fe43acbf67cc1d4c633e88,sentry-release=Unknown"},"correlation_id":"01M26CZJ295MAHNN9NZ2RG7S4B","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:af25898a4e4380d4672aa4894e0ab2d8b944b16ae68493f0501e8456d9bc2527","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:47.726Z","job_size_bytes":2592,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-ba51e23a0c05ca29472c631f: done: 0.019701 sec","job_status":"done","queue_duration_s":0.07807,"scheduling_latency_s":0.07809,"redis_calls":7,"redis_duration_s":0.0014789999999999998,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001099,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00038,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013343,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019701,"completed_at":"2026-09-10T19:33:47.824Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00284,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.931279] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:47.826Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"3a9161780426d730ae942ef6","created_at":"2026-09-10T19:33:47.757Z","trace_propagation_headers":{"sentry-trace":"aa851687784a4572aae0ee315aa9bbd2-55cab685f22b4308","baggage":"sentry-trace_id=aa851687784a4572aae0ee315aa9bbd2,sentry-release=Unknown"},"correlation_id":"01M26CY14TA2KV1CF07DG86K3P","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:47.762Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-3a9161780426d730ae942ef6: start","job_status":"start","queue_duration_s":0.064035,"scheduling_latency_s":0.064056} gitlab # [ 268.938137] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:47.890Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"3a9161780426d730ae942ef6","created_at":"2026-09-10T19:33:47.757Z","trace_propagation_headers":{"sentry-trace":"aa851687784a4572aae0ee315aa9bbd2-55cab685f22b4308","baggage":"sentry-trace_id=aa851687784a4572aae0ee315aa9bbd2,sentry-release=Unknown"},"correlation_id":"01M26CY14TA2KV1CF07DG86K3P","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:47.762Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-3a9161780426d730ae942ef6: done: 0.064091 sec","job_status":"done","queue_duration_s":0.064035,"scheduling_latency_s":0.064056,"redis_calls":11,"redis_duration_s":0.002431,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000457,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001575,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000399,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04643,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000382731,"duration_s":0.064091,"completed_at":"2026-09-10T19:33:47.890Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013522,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 269.157223] gitaly[931]: time="2026-09-10T19:33:48.913Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M26CZR51RHVKKE8Z9KDTRT95 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:48.901" grpc.time_ms=10.951 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 269.159802] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:48 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 80 gitlab # [ 269.219380] gitaly[931]: time="2026-09-10T19:33:48.975Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M26CZR7QV4EC8Z317BVYPATP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:48.968" pid=931 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 269.220507] gitaly[931]: time="2026-09-10T19:33:48.975Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZR7QV4EC8Z317BVYPATP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:48.968" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 269.256584] gitaly[931]: time="2026-09-10T19:33:49.013Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZR7QV4EC8Z317BVYPATP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:48.968" grpc.time_ms=44.462 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 269.279704] gitlab-workhorse[830]: time="2026-09-10T19:33:49Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 269.280799] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:49 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 117 gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3cbe2f74443ff0f3ea476d1bcddd263441bc5445' gitlab # [ 269.354414] gitaly[931]: time="2026-09-10T19:33:49.110Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M26CZRBP497M63F763D8CG91 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:49.092" pid=931 remote_ip="2001:db8:1::1" request_sha=5d104a09fd64a95e6f45ce5b2773312f538561cd response_bytes=336 span.kind=server system=grpc gitlab # [ 269.357143] gitaly[931]: time="2026-09-10T19:33:49.111Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1198 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZRBP497M63F763D8CG91 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:49.092" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 269.359803] gitlab-workhorse[830]: time="2026-09-10T19:33:49Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=336 gitlab # [ 269.361902] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:49 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 336 "" "git/2.55.0" 66 gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3cbe2f74443ff0f3ea476d1bcddd263441bc5445', in 0.05 seconds) gitlab # [ 269.364926] gitaly[931]: time="2026-09-10T19:33:49.111Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1198 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZRBP497M63F763D8CG91 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:49.092" grpc.time_ms=19.063 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 269.424979] nixbot[2723]: INFO: ::ffff:127.0.0.1:37058 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3cbe2f74443ff0f3ea476d1bcddd263441bc5445 HTTP/1.1" 200 OK gitlab # [ 269.468364] nixbot[2723]: {"ts": "2026-09-10T19:33:49.224Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"} gitlab # [ 269.530369] gitaly[931]: time="2026-09-10T19:33:49.286Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZRHPFMJ1WXFNHM2CCYSG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:34:19.085" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:49.285" grpc.time_ms=0.666 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 269.542947] gitaly[931]: time="2026-09-10T19:33:49.299Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M26CZRHMWDKQ5QCRX5S93Z8F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:49.288" grpc.time_ms=10.948 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 269.545675] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:49 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 63 gitlab # [ 269.621138] gitaly[931]: time="2026-09-10T19:33:49.377Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M26CZRHPFMJ1WXFNHM2CCYSG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-10T19:33:59.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:33:49.371" grpc.time_ms=6.181 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 269.642426] gitaly[931]: time="2026-09-10T19:33:49.393Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M26CZRMYDW6DNXYN1S2CZGCH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:49.385" pid=931 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 269.643445] gitlab-workhorse[830]: time="2026-09-10T19:33:49Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 269.644276] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:49 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 52 gitlab # [ 269.645259] gitaly[931]: time="2026-09-10T19:33:49.393Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZRMYDW6DNXYN1S2CZGCH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:49.385" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 269.646189] gitaly[931]: time="2026-09-10T19:33:49.394Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZRMYDW6DNXYN1S2CZGCH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:49.385" grpc.time_ms=8.886 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 269.652598] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:49.408Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"4259eaa7cb11e2d8786b84a8","created_at":"2026-09-10T19:33:49.403Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-9738b2da55ee42bd","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:49.406Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4259eaa7cb11e2d8786b84a8: start","job_status":"start","queue_duration_s":0.001623,"scheduling_latency_s":0.001661} gitlab # [ 269.726211] nixbot[2723]: {"ts": "2026-09-10T19:33:49.424Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/b2555a8ee53ad96300d2a6b920912389a68d95c2 \"HTTP/1.1 201 Created\""} gitlab # [ 269.728606] gitaly[931]: time="2026-09-10T19:33:49.478Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZRQKSEWX2W0CE1ZKTQYY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:34:19.276" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:49.476" grpc.time_ms=1.973 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 269.730878] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:49 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/b2555a8ee53ad96300d2a6b920912389a68d95c2 HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 184 gitlab # [ 269.736908] gitaly[931]: time="2026-09-10T19:33:49.493Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M26CZRHPFMJ1WXFNHM2CCYSG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-10T19:33:49.493" grpc.time_ms=0.302 pid=931 span.kind=server system=grpc gitlab # [ 269.760274] gitaly[931]: time="2026-09-10T19:33:49.516Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CZRQKSEWX2W0CE1ZKTQYY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-10T19:33:59.146" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:33:49.508" grpc.time_ms=8.202 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 269.764578] gitaly[931]: time="2026-09-10T19:33:49.520Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M26CZRHPFMJ1WXFNHM2CCYSG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-10T19:33:49.516" grpc.time_ms=2.921 pid=931 span.kind=server system=grpc gitlab # [ 269.790925] gitaly[931]: time="2026-09-10T19:33:49.547Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M26CZRHPFMJ1WXFNHM2CCYSG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-10T19:33:49.546" grpc.time_ms=0.425 pid=931 span.kind=server system=grpc gitlab # [ 269.800038] gitaly[931]: time="2026-09-10T19:33:49.556Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M26CZRHPFMJ1WXFNHM2CCYSG diskcache=81b654e9-2459-48b7-9ef2-5a33676dc525 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-10T19:34:19.265" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:33:49.466" pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 269.800924] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:49 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/b2555a8ee53ad96300d2a6b920912389a68d95c2 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 129 gitlab # [ 269.804657] gitaly[931]: time="2026-09-10T19:33:49.558Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=78 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7077 command.oublock=0 command.real_time_ms=84 command.spawn_token_fork_ms=1 command.system_time_ms=58 command.user_time_ms=20 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZRHPFMJ1WXFNHM2CCYSG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-10T19:34:19.265" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:33:49.466" grpc.time_ms=90.088 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 269.805916] nixbot[2723]: {"ts": "2026-09-10T19:33:49.558Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/b2555a8ee53ad96300d2a6b920912389a68d95c2 \"HTTP/1.1 201 Created\""} gitlab # [ 269.847041] gitaly[931]: time="2026-09-10T19:33:49.603Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CZRHPFMJ1WXFNHM2CCYSG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-10T19:34:19.401" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:49.602" grpc.time_ms=0.883 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 269.854680] gitaly[931]: time="2026-09-10T19:33:49.611Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZRVXVX6WB06A93VKZV0R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:34:19.409" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:49.610" grpc.time_ms=1.092 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 269.878352] gitaly[931]: time="2026-09-10T19:33:49.634Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZRHPFMJ1WXFNHM2CCYSG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:34:19.433" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:49.634" grpc.time_ms=0.695 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 269.891045] gitaly[931]: time="2026-09-10T19:33:49.647Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M26CZRVXVX6WB06A93VKZV0R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-10T19:33:59.090" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:33:49.640" grpc.time_ms=6.88 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 269.929135] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:49 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/b2555a8ee53ad96300d2a6b920912389a68d95c2 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 117 gitlab # [ 269.930465] nixbot[2723]: {"ts": "2026-09-10T19:33:49.684Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/b2555a8ee53ad96300d2a6b920912389a68d95c2 \"HTTP/1.1 201 Created\""} gitlab # [ 269.931814] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:49.688Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"c32cdc1db44e6da306ec4cec","created_at":"2026-09-10T19:33:49.686Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-db6e9ea8c7714083","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"4259eaa7cb11e2d8786b84a8","job_size_bytes":3,"pid":2598,"job_status":"deduplicated","message":"PipelineProcessWorker JID-c32cdc1db44e6da306ec4cec: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60} gitlab # [ 269.933465] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:49.689Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"4259eaa7cb11e2d8786b84a8","created_at":"2026-09-10T19:33:49.403Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-9738b2da55ee42bd","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:49.406Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2598,"job_status":"rescheduled","message":"PipelineProcessWorker JID-4259eaa7cb11e2d8786b84a8: rescheduled"} gitlab # [ 269.939349] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:49.695Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"4259eaa7cb11e2d8786b84a8","created_at":"2026-09-10T19:33:49.403Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-9738b2da55ee42bd","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:49.406Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4259eaa7cb11e2d8786b84a8: done: 0.287216 sec","job_status":"done","queue_duration_s":0.001623,"scheduling_latency_s":0.001661,"gitaly_calls":3,"gitaly_duration_s":0.103462,"redis_calls":69,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.014418,"redis_read_bytes":226,"redis_write_bytes":18468,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000537,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000777,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.00318,"redis_queues_read_bytes":24,"redis_queues_write_bytes":12314,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.006322,"redis_queues_metadata_read_bytes":110,"redis_queues_metadata_write_bytes":3546,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000225,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00075,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.00209,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.048,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":18,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.034,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.118364,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.203172191,"duration_s":0.287216,"completed_at":"2026-09-10T19:33:49.695Z","load_balancing_strategy":"primary","db_duration_s":0.048762,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 270.049151] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:49.697Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"23321ee9b4e225f02e2f22d1","created_at":"2026-09-10T19:33:49.459Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-db6e9ea8c7714083","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:49.461Z","job_size_bytes":23,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-23321ee9b4e225f02e2f22d1: start","job_status":"start","queue_duration_s":0.235846,"scheduling_latency_s":0.235874} gitlab # [ 270.050255] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:49.738Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"23321ee9b4e225f02e2f22d1","created_at":"2026-09-10T19:33:49.459Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-db6e9ea8c7714083","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:49.461Z","job_size_bytes":23,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-23321ee9b4e225f02e2f22d1: done: 0.041907 sec","job_status":"done","queue_duration_s":0.235846,"scheduling_latency_s":0.235874,"redis_calls":7,"redis_duration_s":0.0016769999999999999,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001233,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000444,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024499,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.041907,"completed_at":"2026-09-10T19:33:49.738Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013155,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 270.122085] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:49.740Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"040c2d4e133a87c37d29c760","created_at":"2026-09-10T19:33:49.562Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-db6e9ea8c7714083","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:49.567Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-040c2d4e133a87c37d29c760: start","job_status":"start","queue_duration_s":0.173117,"scheduling_latency_s":0.173143} gitlab # [ 270.133217] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:49.762Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"040c2d4e133a87c37d29c760","created_at":"2026-09-10T19:33:49.562Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-db6e9ea8c7714083","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:49.567Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-040c2d4e133a87c37d29c760: done: 0.021969 sec","job_status":"done","queue_duration_s":0.173117,"scheduling_latency_s":0.173143,"redis_calls":7,"redis_duration_s":0.001499,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001081,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000418,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021969,"completed_at":"2026-09-10T19:33:49.762Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003117,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 270.219299] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:49.763Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"19268b7111d0efcd45f5ea2a","created_at":"2026-09-10T19:33:49.572Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-db6e9ea8c7714083","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:49.574Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-19268b7111d0efcd45f5ea2a: start","job_status":"start","queue_duration_s":0.188798,"scheduling_latency_s":0.188821} gitlab # [ 270.224340] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:49.785Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"19268b7111d0efcd45f5ea2a","created_at":"2026-09-10T19:33:49.572Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-db6e9ea8c7714083","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:49.574Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-19268b7111d0efcd45f5ea2a: done: 0.021398 sec","job_status":"done","queue_duration_s":0.188798,"scheduling_latency_s":0.188821,"redis_calls":7,"redis_duration_s":0.001422,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001025,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000397,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014454,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021398,"completed_at":"2026-09-10T19:33:49.785Z","load_balancing_strategy":"primary","db_duration_s":0.00346,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 270.295337] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:49.786Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"4c8d1bd7475c1515faa2f7d5","created_at":"2026-09-10T19:33:49.580Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-db6e9ea8c7714083","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:49.582Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-4c8d1bd7475c1515faa2f7d5: start","job_status":"start","queue_duration_s":0.203999,"scheduling_latency_s":0.204021} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3cbe2f74443ff0f3ea476d1bcddd263441bc5445' gitlab # [ 270.375301] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:49.810Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"4c8d1bd7475c1515faa2f7d5","created_at":"2026-09-10T19:33:49.580Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-db6e9ea8c7714083","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:49.582Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-4c8d1bd7475c1515faa2f7d5: done: 0.024009 sec","job_status":"done","queue_duration_s":0.203999,"scheduling_latency_s":0.204021,"redis_calls":10,"redis_duration_s":0.0020150000000000003,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001056,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00058,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000379,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016636,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024009,"completed_at":"2026-09-10T19:33:49.810Z","load_balancing_strategy":"primary","db_duration_s":0.00345,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3cbe2f74443ff0f3ea476d1bcddd263441bc5445', in 0.03 seconds) gitlab # [ 270.382799] gitaly[931]: time="2026-09-10T19:33:50.000Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M26CZRHPFMJ1WXFNHM2CCYSG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-10T19:33:50.000" grpc.time_ms=0.313 pid=931 span.kind=server system=grpc gitlab # [ 270.463831] gitaly[931]: time="2026-09-10T19:33:50.020Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M26CZRHPFMJ1WXFNHM2CCYSG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-10T19:33:50.020" grpc.time_ms=0.261 pid=931 span.kind=server system=grpc gitlab # [ 270.464944] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:49.811Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"ffe6fad03c45314f2d70daef","created_at":"2026-09-10T19:33:49.582Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-db6e9ea8c7714083","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:49.584Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-ffe6fad03c45314f2d70daef: start","job_status":"start","queue_duration_s":0.227431,"scheduling_latency_s":0.227453} gitlab # [ 270.465747] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:49.832Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"ffe6fad03c45314f2d70daef","created_at":"2026-09-10T19:33:49.582Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-db6e9ea8c7714083","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:49.584Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-ffe6fad03c45314f2d70daef: done: 0.020317 sec","job_status":"done","queue_duration_s":0.227431,"scheduling_latency_s":0.227453,"redis_calls":7,"redis_duration_s":0.001499,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001076,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000423,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013557,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020317,"completed_at":"2026-09-10T19:33:49.832Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003127,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 270.549492] nixbot[2723]: INFO: ::ffff:127.0.0.1:37070 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3cbe2f74443ff0f3ea476d1bcddd263441bc5445 HTTP/1.1" 200 OK gitlab # [ 270.550354] gitaly[931]: time="2026-09-10T19:33:50.041Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M26CZRHPFMJ1WXFNHM2CCYSG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-10T19:33:50.040" grpc.time_ms=0.257 pid=931 span.kind=server system=grpc gitlab # [ 270.551172] gitaly[931]: time="2026-09-10T19:33:50.053Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M26CZRHPFMJ1WXFNHM2CCYSG diskcache=b63a6a58-7e56-4c6a-a4e5-bf2361f60c89 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-10T19:34:19.777" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:33:49.977" pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 270.552064] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:49.833Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"25d8a60ab24253e7ac019b51","created_at":"2026-09-10T19:33:49.584Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-db6e9ea8c7714083","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:49.587Z","job_size_bytes":19,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-25d8a60ab24253e7ac019b51: start","job_status":"start","queue_duration_s":0.246279,"scheduling_latency_s":0.2463} gitlab # [ 270.562239] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:49.852Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"25d8a60ab24253e7ac019b51","created_at":"2026-09-10T19:33:49.584Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-db6e9ea8c7714083","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:49.587Z","job_size_bytes":19,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-25d8a60ab24253e7ac019b51: done: 0.019103 sec","job_status":"done","queue_duration_s":0.246279,"scheduling_latency_s":0.2463,"redis_calls":8,"redis_duration_s":0.00174,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001359,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000381,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012563,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019103,"completed_at":"2026-09-10T19:33:49.852Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002348,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 270.628395] gitaly[931]: time="2026-09-10T19:33:50.053Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=77 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7388 command.oublock=0 command.real_time_ms=67 command.spawn_token_fork_ms=1 command.system_time_ms=46 command.user_time_ms=31 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZRHPFMJ1WXFNHM2CCYSG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-10T19:34:19.777" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:33:49.977" grpc.time_ms=75.73 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 270.699690] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:49.853Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"3a6e56233483a12031b9bb7d","created_at":"2026-09-10T19:33:49.587Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-db6e9ea8c7714083","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:49.589Z","job_size_bytes":66,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-3a6e56233483a12031b9bb7d: start","job_status":"start","queue_duration_s":0.264696,"scheduling_latency_s":0.264719} gitlab # [ 270.700793] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:49.871Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"3a6e56233483a12031b9bb7d","created_at":"2026-09-10T19:33:49.587Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-db6e9ea8c7714083","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:49.589Z","job_size_bytes":66,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-3a6e56233483a12031b9bb7d: done: 0.018041 sec","job_status":"done","queue_duration_s":0.264696,"scheduling_latency_s":0.264719,"redis_calls":7,"redis_duration_s":0.0014420000000000001,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001062,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00038,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012257,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018041,"completed_at":"2026-09-10T19:33:49.871Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001882,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 270.742064] gitaly[931]: time="2026-09-10T19:33:50.095Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CZRHPFMJ1WXFNHM2CCYSG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-10T19:34:20.094" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:50.094" grpc.time_ms=0.679 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 270.761287] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:49.873Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"2ccede2a48a79455dc298424","created_at":"2026-09-10T19:33:49.589Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-db6e9ea8c7714083","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:49.590Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-2ccede2a48a79455dc298424: start","job_status":"start","queue_duration_s":0.282256,"scheduling_latency_s":0.282277} gitlab # [ 270.839118] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:49.894Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"2ccede2a48a79455dc298424","created_at":"2026-09-10T19:33:49.589Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-db6e9ea8c7714083","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:49.590Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-2ccede2a48a79455dc298424: done: 0.020864 sec","job_status":"done","queue_duration_s":0.282256,"scheduling_latency_s":0.282277,"redis_calls":7,"redis_duration_s":0.0014990000000000001,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001072,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000427,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013195,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020864,"completed_at":"2026-09-10T19:33:49.893Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003931,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 270.851838] gitaly[931]: time="2026-09-10T19:33:50.117Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZRHPFMJ1WXFNHM2CCYSG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:34:20.115" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:50.116" grpc.time_ms=0.588 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 270.871143] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:49.895Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"4f8f2a3159146de710624ba9","created_at":"2026-09-10T19:33:49.591Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-db6e9ea8c7714083","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:49.592Z","job_size_bytes":6,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-4f8f2a3159146de710624ba9: start","job_status":"start","queue_duration_s":0.302826,"scheduling_latency_s":0.302848} gitlab # [ 270.895345] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:49.907Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"4f8f2a3159146de710624ba9","created_at":"2026-09-10T19:33:49.591Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-db6e9ea8c7714083","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:49.592Z","job_size_bytes":6,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-4f8f2a3159146de710624ba9: done: 0.012602 sec","job_status":"done","queue_duration_s":0.302826,"scheduling_latency_s":0.302848,"redis_calls":7,"redis_duration_s":0.001418,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000998,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00042,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008197,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012602,"completed_at":"2026-09-10T19:33:49.907Z","load_balancing_strategy":"primary","db_duration_s":0.000332,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 270.980055] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:49.909Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"88373b39ef00631c9521ae72","created_at":"2026-09-10T19:33:49.593Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-db6e9ea8c7714083","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:49.594Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-88373b39ef00631c9521ae72: start","job_status":"start","queue_duration_s":0.314997,"scheduling_latency_s":0.315018} gitlab # [ 270.983088] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:49.927Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"88373b39ef00631c9521ae72","created_at":"2026-09-10T19:33:49.593Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-db6e9ea8c7714083","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:49.594Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-88373b39ef00631c9521ae72: done: 0.018606 sec","job_status":"done","queue_duration_s":0.314997,"scheduling_latency_s":0.315018,"redis_calls":7,"redis_duration_s":0.001441,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001055,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000386,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012427,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018606,"completed_at":"2026-09-10T19:33:49.927Z","load_balancing_strategy":"primary","db_duration_s":0.002531,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 271.106384] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:49.929Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"6793b42e805dc4f84311bc68","created_at":"2026-09-10T19:33:49.690Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-db6e9ea8c7714083","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:49.691Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-6793b42e805dc4f84311bc68: start","job_status":"start","queue_duration_s":0.2379,"scheduling_latency_s":0.237921} gitlab # [ 271.107834] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:50.126Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"6793b42e805dc4f84311bc68","created_at":"2026-09-10T19:33:49.690Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-db6e9ea8c7714083","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:49.691Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-6793b42e805dc4f84311bc68: done: 0.197356 sec","job_status":"done","queue_duration_s":0.2379,"scheduling_latency_s":0.237921,"gitaly_calls":3,"gitaly_duration_s":0.083008,"redis_calls":63,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.013602,"redis_read_bytes":141,"redis_write_bytes":16719,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000432,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.00077,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":20,"redis_queues_duration_s":0.00326,"redis_queues_read_bytes":21,"redis_queues_write_bytes":11225,"redis_queues_metadata_calls":18,"redis_queues_metadata_duration_s":0.006215,"redis_queues_metadata_read_bytes":28,"redis_queues_metadata_write_bytes":2886,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000209,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000582,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001702,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.085763,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.171858714,"duration_s":0.197356,"completed_at":"2026-09-10T19:33:50.126Z","load_balancing_strategy":"primary","db_duration_s":0.021673,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 271.150221] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:50.127Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"e9bbc7a12e0486b102d79a23","created_at":"2026-09-10T19:33:49.971Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-df48f46593f34810","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:49.973Z","job_size_bytes":23,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-e9bbc7a12e0486b102d79a23: start","job_status":"start","queue_duration_s":0.154896,"scheduling_latency_s":0.154918} gitlab # [ 271.175907] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:50.167Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"e9bbc7a12e0486b102d79a23","created_at":"2026-09-10T19:33:49.971Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-df48f46593f34810","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:49.973Z","job_size_bytes":23,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-e9bbc7a12e0486b102d79a23: done: 0.039693 sec","job_status":"done","queue_duration_s":0.154896,"scheduling_latency_s":0.154918,"redis_calls":7,"redis_duration_s":0.006057,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003029,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003028,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023424,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.039693,"completed_at":"2026-09-10T19:33:50.167Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008199,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 271.260282] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:50.169Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"c2971080bbf47370e2b18154","created_at":"2026-09-10T19:33:50.058Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-df48f46593f34810","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:50.062Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-c2971080bbf47370e2b18154: start","job_status":"start","queue_duration_s":0.106054,"scheduling_latency_s":0.106077} gitlab # [ 271.266297] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:50.190Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"c2971080bbf47370e2b18154","created_at":"2026-09-10T19:33:50.058Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-df48f46593f34810","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:50.062Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-c2971080bbf47370e2b18154: done: 0.021181 sec","job_status":"done","queue_duration_s":0.106054,"scheduling_latency_s":0.106077,"redis_calls":7,"redis_duration_s":0.0014650000000000002,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001064,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000401,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014485,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021181,"completed_at":"2026-09-10T19:33:50.190Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002842,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 271.333341] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:50.191Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"30eae8501e731e84dfb4c153","created_at":"2026-09-10T19:33:50.068Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-df48f46593f34810","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:50.069Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-30eae8501e731e84dfb4c153: start","job_status":"start","queue_duration_s":0.122123,"scheduling_latency_s":0.122146} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3cbe2f74443ff0f3ea476d1bcddd263441bc5445' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3cbe2f74443ff0f3ea476d1bcddd263441bc5445', in 0.03 seconds) gitlab # [ 271.400461] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:50.216Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"30eae8501e731e84dfb4c153","created_at":"2026-09-10T19:33:50.068Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-df48f46593f34810","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:50.069Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-30eae8501e731e84dfb4c153: done: 0.024723 sec","job_status":"done","queue_duration_s":0.122123,"scheduling_latency_s":0.122146,"redis_calls":7,"redis_duration_s":0.001558,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001095,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000463,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015163,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024723,"completed_at":"2026-09-10T19:33:50.216Z","load_balancing_strategy":"primary","db_duration_s":0.005063,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 271.475449] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:50.217Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"e80531731c0698ab895c7859","created_at":"2026-09-10T19:33:50.075Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-df48f46593f34810","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:50.076Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-e80531731c0698ab895c7859: start","job_status":"start","queue_duration_s":0.141507,"scheduling_latency_s":0.141529} gitlab # [ 271.476880] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:50.243Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"e80531731c0698ab895c7859","created_at":"2026-09-10T19:33:50.075Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-df48f46593f34810","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:50.076Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-e80531731c0698ab895c7859: done: 0.025395 sec","job_status":"done","queue_duration_s":0.141507,"scheduling_latency_s":0.141529,"redis_calls":10,"redis_duration_s":0.002022,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001056,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000584,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000382,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016857,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025395,"completed_at":"2026-09-10T19:33:50.243Z","load_balancing_strategy":"primary","db_duration_s":0.003396,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 271.568091] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:50.244Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"5622ad47607b555ce2a5c687","created_at":"2026-09-10T19:33:50.076Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-df48f46593f34810","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:50.077Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-5622ad47607b555ce2a5c687: start","job_status":"start","queue_duration_s":0.166553,"scheduling_latency_s":0.166575} gitlab # [ 271.569242] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:50.264Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"5622ad47607b555ce2a5c687","created_at":"2026-09-10T19:33:50.076Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-df48f46593f34810","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:50.077Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-5622ad47607b555ce2a5c687: done: 0.020367 sec","job_status":"done","queue_duration_s":0.166553,"scheduling_latency_s":0.166575,"redis_calls":7,"redis_duration_s":0.0014689999999999998,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001055,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000414,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01354,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020367,"completed_at":"2026-09-10T19:33:50.264Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003054,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 271.638735] nixbot[2723]: INFO: ::ffff:127.0.0.1:37080 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3cbe2f74443ff0f3ea476d1bcddd263441bc5445 HTTP/1.1" 200 OK gitlab # [ 271.639864] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:50.266Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"4e19d5b7d9cf5dcf03a3f8ab","created_at":"2026-09-10T19:33:50.078Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-df48f46593f34810","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:50.080Z","job_size_bytes":19,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-4e19d5b7d9cf5dcf03a3f8ab: start","job_status":"start","queue_duration_s":0.185894,"scheduling_latency_s":0.185916} gitlab # [ 271.640929] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:50.285Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"4e19d5b7d9cf5dcf03a3f8ab","created_at":"2026-09-10T19:33:50.078Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-df48f46593f34810","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:50.080Z","job_size_bytes":19,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-4e19d5b7d9cf5dcf03a3f8ab: done: 0.019061 sec","job_status":"done","queue_duration_s":0.185894,"scheduling_latency_s":0.185916,"redis_calls":8,"redis_duration_s":0.0016829999999999998,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001298,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000385,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012504,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019061,"completed_at":"2026-09-10T19:33:50.285Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002501,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 271.720603] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:50.286Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"46822a05a97d822bc3ea4b63","created_at":"2026-09-10T19:33:50.081Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-df48f46593f34810","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:50.082Z","job_size_bytes":66,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-46822a05a97d822bc3ea4b63: start","job_status":"start","queue_duration_s":0.204353,"scheduling_latency_s":0.204375} gitlab # [ 271.724842] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:50.304Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"46822a05a97d822bc3ea4b63","created_at":"2026-09-10T19:33:50.081Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-df48f46593f34810","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:50.082Z","job_size_bytes":66,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-46822a05a97d822bc3ea4b63: done: 0.017964 sec","job_status":"done","queue_duration_s":0.204353,"scheduling_latency_s":0.204375,"redis_calls":7,"redis_duration_s":0.0014939999999999999,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001084,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00041,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012182,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017964,"completed_at":"2026-09-10T19:33:50.304Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001933,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 271.820430] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:50.305Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"9bfa595abf642a02c4712768","created_at":"2026-09-10T19:33:50.082Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-df48f46593f34810","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:50.083Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-9bfa595abf642a02c4712768: start","job_status":"start","queue_duration_s":0.22206,"scheduling_latency_s":0.22208} gitlab # [ 271.821737] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:50.327Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"9bfa595abf642a02c4712768","created_at":"2026-09-10T19:33:50.082Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-df48f46593f34810","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:50.083Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-9bfa595abf642a02c4712768: done: 0.02141 sec","job_status":"done","queue_duration_s":0.22206,"scheduling_latency_s":0.22208,"redis_calls":7,"redis_duration_s":0.0014680000000000001,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001067,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000401,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013355,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02141,"completed_at":"2026-09-10T19:33:50.327Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004317,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 271.883541] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:50.328Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"8ec863d13d06287fea98b0b2","created_at":"2026-09-10T19:33:50.084Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-df48f46593f34810","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:50.085Z","job_size_bytes":6,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-8ec863d13d06287fea98b0b2: start","job_status":"start","queue_duration_s":0.243303,"scheduling_latency_s":0.243369} gitlab # [ 271.960138] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:50.341Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"8ec863d13d06287fea98b0b2","created_at":"2026-09-10T19:33:50.084Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-df48f46593f34810","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:50.085Z","job_size_bytes":6,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-8ec863d13d06287fea98b0b2: done: 0.012719 sec","job_status":"done","queue_duration_s":0.243303,"scheduling_latency_s":0.243369,"redis_calls":7,"redis_duration_s":0.001467,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001077,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00039,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008205,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012719,"completed_at":"2026-09-10T19:33:50.341Z","load_balancing_strategy":"primary","db_duration_s":0.000373,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 271.971158] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:50.342Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"392cb4d1898940aed2d0a2b0","created_at":"2026-09-10T19:33:50.085Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-df48f46593f34810","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:50.086Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-392cb4d1898940aed2d0a2b0: start","job_status":"start","queue_duration_s":0.255839,"scheduling_latency_s":0.25586} gitlab # [ 272.057136] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:50.361Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"392cb4d1898940aed2d0a2b0","created_at":"2026-09-10T19:33:50.085Z","trace_propagation_headers":{"sentry-trace":"7d42a66400f0452e9a5a4aa0cbb7be4c-df48f46593f34810","baggage":"sentry-trace_id=7d42a66400f0452e9a5a4aa0cbb7be4c,sentry-release=Unknown"},"correlation_id":"01M26CZRHPFMJ1WXFNHM2CCYSG","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:50.086Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-392cb4d1898940aed2d0a2b0: done: 0.018779 sec","job_status":"done","queue_duration_s":0.255839,"scheduling_latency_s":0.25586,"redis_calls":7,"redis_duration_s":0.001485,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001106,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000379,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012364,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018779,"completed_at":"2026-09-10T19:33:50.361Z","load_balancing_strategy":"primary","db_duration_s":0.00249,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 272.193339] gitaly[931]: time="2026-09-10T19:33:51.949Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M26CZV4WHBBWCWEKRD7P731Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:51.942" grpc.time_ms=6.508 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 272.194724] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:51 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 49 gitlab # [ 272.217711] gitaly[931]: time="2026-09-10T19:33:51.974Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CZV51GE5J97ZPRT64M9W1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:51.945" grpc.time_ms=28.162 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 272.239597] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:51 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 69 gitlab # [ 272.251095] gitaly[931]: time="2026-09-10T19:33:52.007Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M26CZV6GK07T97RJVJ9GFZ6C grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:51.996" pid=931 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 272.252206] gitaly[931]: time="2026-09-10T19:33:52.007Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZV6GK07T97RJVJ9GFZ6C grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:51.996" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 272.253284] gitlab-workhorse[830]: time="2026-09-10T19:33:52Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 272.257339] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:52 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 55 gitlab # [ 272.258963] gitaly[931]: time="2026-09-10T19:33:52.007Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZV6GK07T97RJVJ9GFZ6C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:51.996" grpc.time_ms=11.384 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 272.286587] gitaly[931]: time="2026-09-10T19:33:52.043Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M26CZV78TTHSHJKBQQERGCTZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:52.027" pid=931 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc gitlab # [ 272.287791] gitaly[931]: time="2026-09-10T19:33:52.043Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1169 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZV78TTHSHJKBQQERGCTZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:52.027" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 272.288852] gitaly[931]: time="2026-09-10T19:33:52.043Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1169 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZV78TTHSHJKBQQERGCTZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:33:52.027" grpc.time_ms=16.447 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 272.371136] gitlab-workhorse[830]: time="2026-09-10T19:33:52Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=216 gitlab # [ 272.372351] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:52 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 68 gitlab # [ 272.449344] gitaly[931]: time="2026-09-10T19:33:52.205Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZVD34KJSZDD9CD3GKKMV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:34:22.104" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:52.205" grpc.time_ms=0.477 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3cbe2f74443ff0f3ea476d1bcddd263441bc5445' gitlab # [ 272.461959] gitaly[931]: time="2026-09-10T19:33:52.218Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=651 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M26CZVD34KJSZDD9CD3GKKMV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-10T19:34:22.112" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:33:52.212" grpc.time_ms=5.507 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 272.501262] gitaly[931]: time="2026-09-10T19:33:52.257Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CZVD34KJSZDD9CD3GKKMV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-10T19:34:02.112" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:33:52.252" grpc.time_ms=5.202 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 272.508897] nixbot[2723]: INFO: ::ffff:127.0.0.1:37094 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3cbe2f74443ff0f3ea476d1bcddd263441bc5445 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3cbe2f74443ff0f3ea476d1bcddd263441bc5445', in 0.04 seconds) (finished: subtest: push delivers the webhook, in 54.67 seconds) subtest: eval and build post commit statuses gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses gitlab # [ 272.548982] gitaly[931]: time="2026-09-10T19:33:52.305Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M26CZVD34KJSZDD9CD3GKKMV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-10T19:34:02.164" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:33:52.305" grpc.time_ms=0.215 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 272.576677] gitaly[931]: time="2026-09-10T19:33:52.333Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZVD34KJSZDD9CD3GKKMV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-10T19:34:02.186" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:33:52.327" grpc.time_ms=5.907 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 272.598478] gitaly[931]: time="2026-09-10T19:33:52.337Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZVH9929C1ZR9F4E71YDJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:34:22.136" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:52.337" grpc.time_ms=0.645 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 272.610968] gitaly[931]: time="2026-09-10T19:33:52.367Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M26CZVD34KJSZDD9CD3GKKMV grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-10T19:33:52.366" grpc.time_ms=0.446 pid=931 span.kind=server system=grpc gitlab # [ 272.631847] gitaly[931]: time="2026-09-10T19:33:52.388Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M26CZVD34KJSZDD9CD3GKKMV grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-10T19:33:52.387" grpc.time_ms=0.266 pid=931 span.kind=server system=grpc gitlab # [ 272.636776] gitaly[931]: time="2026-09-10T19:33:52.393Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M26CZVD34KJSZDD9CD3GKKMV forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-10T19:34:02.196" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:33:52.337" operations="[repack_refs]" pid=931 ref=refs/keep-around/3cbe2f74443ff0f3ea476d1bcddd263441bc5445 remote_ip="2001:db8:1::1" revision=3cbe2f74443ff0f3ea476d1bcddd263441bc5445 span.kind=server system=grpc username=root gitlab # [ 272.638395] gitaly[931]: time="2026-09-10T19:33:52.394Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M26CZVD34KJSZDD9CD3GKKMV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-10T19:34:02.196" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:33:52.337" pid=931 ref=refs/keep-around/3cbe2f74443ff0f3ea476d1bcddd263441bc5445 remote_ip="2001:db8:1::1" repository_info="{false {6 703 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-09-10 19:32:45.851724288 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" revision=3cbe2f74443ff0f3ea476d1bcddd263441bc5445 span.kind=server system=grpc username=root gitlab # [ 272.643393] gitaly[931]: time="2026-09-10T19:33:52.399Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M26CZVD34KJSZDD9CD3GKKMV diskcache=9d909ff6-97be-48a8-9c09-0ce216a45589 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-10T19:34:02.196" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:33:52.337" pid=931 ref=refs/keep-around/3cbe2f74443ff0f3ea476d1bcddd263441bc5445 remote_ip="2001:db8:1::1" revision=3cbe2f74443ff0f3ea476d1bcddd263441bc5445 span.kind=server system=grpc username=root gitlab # [ 272.644488] gitaly[931]: time="2026-09-10T19:33:52.400Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=56 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5574 command.oublock=8 command.real_time_ms=57 command.spawn_token_fork_ms=0 command.system_time_ms=41 command.user_time_ms=15 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZVD34KJSZDD9CD3GKKMV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-10T19:34:02.196" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:33:52.337" grpc.time_ms=62.896 pid=931 ref=refs/keep-around/3cbe2f74443ff0f3ea476d1bcddd263441bc5445 remote_ip="2001:db8:1::1" revision=3cbe2f74443ff0f3ea476d1bcddd263441bc5445 span.kind=server system=grpc username=root gitlab # [ 272.660192] gitaly[931]: time="2026-09-10T19:33:52.406Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZVD34KJSZDD9CD3GKKMV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:34:22.103" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:52.404" grpc.time_ms=1.954 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 272.679734] gitaly[931]: time="2026-09-10T19:33:52.410Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M26CZVD34KJSZDD9CD3GKKMV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-10T19:34:02.196" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:33:52.337" pid=931 ref=refs/keep-around/3cbe2f74443ff0f3ea476d1bcddd263441bc5445 remote_ip="2001:db8:1::1" revision=3cbe2f74443ff0f3ea476d1bcddd263441bc5445 span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root gitlab # [ 272.703137] gitaly[931]: time="2026-09-10T19:33:52.410Z" level=info msg="optimized repository" clean_stale_data_time=14.769754ms clean_work_trees_time="89.956µs" commit_graph_time="8.101µs" correlation_id=01M26CZVD34KJSZDD9CD3GKKMV gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="8.381µs" pid=931 prune_time="7.822µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="8.381µs" storage_name=default system=housekeeping total_time=15.221767ms gitlab # [ 272.712323] gitaly[931]: time="2026-09-10T19:33:52.410Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M26CZVD34KJSZDD9CD3GKKMV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-10T19:34:02.196" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-10T19:33:52.337" pid=931 ref=refs/keep-around/3cbe2f74443ff0f3ea476d1bcddd263441bc5445 remote_ip="2001:db8:1::1" revision=3cbe2f74443ff0f3ea476d1bcddd263441bc5445 span.kind=server system=grpc username=root gitlab # [ 272.733265] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:52.489Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"24eed394ff79df578344db11","created_at":"2026-09-10T19:33:52.483Z","trace_propagation_headers":{"sentry-trace":"496100434f8e4dd79250d0e7702df084-8338dd19a0e0431f","baggage":"sentry-trace_id=496100434f8e4dd79250d0e7702df084,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CZVD34KJSZDD9CD3GKKMV","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:52.487Z","job_size_bytes":65,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-24eed394ff79df578344db11: start","job_status":"start","queue_duration_s":0.001719,"scheduling_latency_s":0.00176} gitlab # [ 272.808549] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:52.519Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"24eed394ff79df578344db11","created_at":"2026-09-10T19:33:52.483Z","trace_propagation_headers":{"sentry-trace":"496100434f8e4dd79250d0e7702df084-8338dd19a0e0431f","baggage":"sentry-trace_id=496100434f8e4dd79250d0e7702df084,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CZVD34KJSZDD9CD3GKKMV","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:52.487Z","job_size_bytes":65,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-24eed394ff79df578344db11: done: 0.030302 sec","job_status":"done","queue_duration_s":0.001719,"scheduling_latency_s":0.00176,"redis_calls":7,"redis_duration_s":0.001765,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001288,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000477,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017465,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030302,"completed_at":"2026-09-10T19:33:52.519Z","load_balancing_strategy":"primary","db_duration_s":0.007074,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 272.881591] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:52.520Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"9580644cc17e7acf275e1b59","created_at":"2026-09-10T19:33:52.488Z","trace_propagation_headers":{"sentry-trace":"496100434f8e4dd79250d0e7702df084-8338dd19a0e0431f","baggage":"sentry-trace_id=496100434f8e4dd79250d0e7702df084,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CZVD34KJSZDD9CD3GKKMV","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:52.490Z","job_size_bytes":65,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-9580644cc17e7acf275e1b59: start","job_status":"start","queue_duration_s":0.030474,"scheduling_latency_s":0.030501} gitlab # [ 272.882923] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:52.572Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"9580644cc17e7acf275e1b59","created_at":"2026-09-10T19:33:52.488Z","trace_propagation_headers":{"sentry-trace":"496100434f8e4dd79250d0e7702df084-8338dd19a0e0431f","baggage":"sentry-trace_id=496100434f8e4dd79250d0e7702df084,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CZVD34KJSZDD9CD3GKKMV","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:52.490Z","job_size_bytes":65,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-9580644cc17e7acf275e1b59: done: 0.051262 sec","job_status":"done","queue_duration_s":0.030474,"scheduling_latency_s":0.030501,"redis_calls":10,"redis_duration_s":0.0046,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002385,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.002215,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.051262,"completed_at":"2026-09-10T19:33:52.572Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01645,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 272.969180] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:52.573Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"d4afc08eec1a26ae41682e20","created_at":"2026-09-10T19:33:52.551Z","trace_propagation_headers":{"sentry-trace":"496100434f8e4dd79250d0e7702df084-8338dd19a0e0431f","baggage":"sentry-trace_id=496100434f8e4dd79250d0e7702df084,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CZVD34KJSZDD9CD3GKKMV","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:52.554Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d4afc08eec1a26ae41682e20: start","job_status":"start","queue_duration_s":0.018612,"scheduling_latency_s":0.018639} gitlab # [ 272.970513] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:52 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/3cbe2f74443ff0f3ea476d1bcddd263441bc5445 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 403 gitlab # [ 272.972394] gitaly[931]: time="2026-09-10T19:33:52.637Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26CZVD34KJSZDD9CD3GKKMV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-10T19:34:22.435" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:52.636" grpc.time_ms=1.007 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 273.021656] nixbot[2723]: {"ts": "2026-09-10T19:33:52.568Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/3cbe2f74443ff0f3ea476d1bcddd263441bc5445 \"HTTP/1.1 201 Created\""} gitlab # [ 273.022920] nixbot[2723]: {"ts": "2026-09-10T19:33:52.576Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-3cbe2f74443f-4"} gitlab # [ 273.023719] nixbot[2723]: {"ts": "2026-09-10T19:33:52.671Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-3cbe2f74443f-4", "argv0": "bwrap"} gitlab # [ 273.025043] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:52.683Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"d4afc08eec1a26ae41682e20","created_at":"2026-09-10T19:33:52.551Z","trace_propagation_headers":{"sentry-trace":"496100434f8e4dd79250d0e7702df084-8338dd19a0e0431f","baggage":"sentry-trace_id=496100434f8e4dd79250d0e7702df084,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M26CZVD34KJSZDD9CD3GKKMV","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:52.554Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d4afc08eec1a26ae41682e20: done: 0.109513 sec","job_status":"done","queue_duration_s":0.018612,"scheduling_latency_s":0.018639,"gitaly_calls":2,"gitaly_duration_s":0.009635,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.007131,"redis_read_bytes":79,"redis_write_bytes":5122,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.00111,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001062,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000272,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001731,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000277,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000445,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001124,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.072507,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.082245648,"duration_s":0.109513,"completed_at":"2026-09-10T19:33:52.682Z","load_balancing_strategy":"primary","db_duration_s":0.024161,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 273.109898] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:52.684Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"e551768bfdfd1bd7bc36bb79","created_at":"2026-09-10T19:33:52.622Z","trace_propagation_headers":{"sentry-trace":"496100434f8e4dd79250d0e7702df084-f7818f0faaef48d8","baggage":"sentry-trace_id=496100434f8e4dd79250d0e7702df084,sentry-release=Unknown"},"correlation_id":"01M26CZVD34KJSZDD9CD3GKKMV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:52.624Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-e551768bfdfd1bd7bc36bb79: start","job_status":"start","queue_duration_s":0.059647,"scheduling_latency_s":0.059669} gitlab # [ 273.110961] gitaly[931]: time="2026-09-10T19:33:52.673Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZVD34KJSZDD9CD3GKKMV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:34:22.471" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:52.672" grpc.time_ms=0.619 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 273.112150] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:52.745Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"e551768bfdfd1bd7bc36bb79","created_at":"2026-09-10T19:33:52.622Z","trace_propagation_headers":{"sentry-trace":"496100434f8e4dd79250d0e7702df084-f7818f0faaef48d8","baggage":"sentry-trace_id=496100434f8e4dd79250d0e7702df084,sentry-release=Unknown"},"correlation_id":"01M26CZVD34KJSZDD9CD3GKKMV","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:52.624Z","job_size_bytes":3,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-e551768bfdfd1bd7bc36bb79: done: 0.061132 sec","job_status":"done","queue_duration_s":0.059647,"scheduling_latency_s":0.059669,"redis_calls":7,"redis_duration_s":0.0015010000000000002,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001103,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000398,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.036,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.034,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020774,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.061132,"completed_at":"2026-09-10T19:33:52.745Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.036124,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.231676] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:52 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses HTTP/1.1" 200 2 "" "curl/8.21.0" 689 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses, in 0.72 seconds) gitlab # [ 273.284890] nix-daemon[2886]: accepted connection from pid 3771, user nixbot gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses gitlab # [ 274.350265] gitaly[931]: time="2026-09-10T19:33:54.105Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZX7SM01WVC2N0D5H6QAJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:34:24.003" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:54.104" grpc.time_ms=0.995 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 274.370818] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:54 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 84 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses, in 0.16 seconds) gitlab # [ 274.710968] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:54.465Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"1ae534aa915e97cfc18850da","created_at":"2026-09-10T19:33:54.445Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"400bed76a2c3d2388ebecc751e610f71","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:54.463Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-1ae534aa915e97cfc18850da: start","job_status":"start","queue_duration_s":0.002428,"scheduling_latency_s":0.002605} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses, in 0.10 seconds) gitlab # [ 275.485940] gitaly[931]: time="2026-09-10T19:33:55.241Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZYC071J28VTH74FDWF0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:34:25.040" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:55.241" grpc.time_ms=0.783 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 275.501921] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:55 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 57 gitlab # [ 275.549628] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:55.305Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"1ae534aa915e97cfc18850da","created_at":"2026-09-10T19:33:54.445Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"400bed76a2c3d2388ebecc751e610f71","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:54.463Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-1ae534aa915e97cfc18850da: done: 0.840108 sec","job_status":"done","queue_duration_s":0.002428,"scheduling_latency_s":0.002605,"redis_calls":13,"redis_duration_s":0.0174,"redis_read_bytes":11,"redis_write_bytes":1563,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.011851,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":576,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002503,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003046,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.157,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.152,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.307646,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001274185,"exclusive_lock_hold_duration_s":0.737183357,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.840108,"completed_at":"2026-09-10T19:33:55.305Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.481023,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.628667] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:55.384Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"3120e6f819e2ad3f18599d86","created_at":"2026-09-10T19:33:54.507Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4e5109b3c3bdb8bfe44d49d6157344bb","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:54.518Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-3120e6f819e2ad3f18599d86: start","job_status":"start","queue_duration_s":0.866267,"scheduling_latency_s":0.866301} gitlab # [ 275.794915] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:55.549Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"3120e6f819e2ad3f18599d86","created_at":"2026-09-10T19:33:54.507Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4e5109b3c3bdb8bfe44d49d6157344bb","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:54.518Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-3120e6f819e2ad3f18599d86: done: 0.165284 sec","job_status":"done","queue_duration_s":0.866267,"scheduling_latency_s":0.866301,"redis_calls":9,"redis_duration_s":0.021974,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002899,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.01688,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002195,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.04,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.04,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033322,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.165284,"completed_at":"2026-09-10T19:33:55.549Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.044279,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.805642] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:55.561Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"13f1ca6b27baacda62f1bcc0","created_at":"2026-09-10T19:33:54.541Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8b31f5647394531e5c7c919889f3a1de","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:54.600Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-13f1ca6b27baacda62f1bcc0: start","job_status":"start","queue_duration_s":0.961312,"scheduling_latency_s":0.961368} gitlab # [ 275.923741] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:55.679Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"13f1ca6b27baacda62f1bcc0","created_at":"2026-09-10T19:33:54.541Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8b31f5647394531e5c7c919889f3a1de","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:54.600Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-13f1ca6b27baacda62f1bcc0: done: 0.117955 sec","job_status":"done","queue_duration_s":0.961312,"scheduling_latency_s":0.961368,"redis_calls":11,"redis_duration_s":0.023723,"redis_read_bytes":9,"redis_write_bytes":1509,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003139,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":347,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005903,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.014681,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":7,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043968,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001370007,"exclusive_lock_hold_duration_s":0.048356145,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.117955,"completed_at":"2026-09-10T19:33:55.679Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.038387,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.927419] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:55.682Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"5e6c723041160d45630104b8","created_at":"2026-09-10T19:33:54.620Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2ded6d70b316601dbf5336c8c354a231","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-10T19:33:54.638Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-5e6c723041160d45630104b8: start","job_status":"start","queue_duration_s":1.043879,"scheduling_latency_s":1.043919} gitlab # [ 276.025273] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:55.781Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 276.041598] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:55.797Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"5e6c723041160d45630104b8","created_at":"2026-09-10T19:33:54.620Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2ded6d70b316601dbf5336c8c354a231","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-10T19:33:54.638Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-5e6c723041160d45630104b8: done: 0.11482 sec","job_status":"done","queue_duration_s":1.043879,"scheduling_latency_s":1.043919,"redis_calls":9,"redis_duration_s":0.005688,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001226,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003301,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001161,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.074,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.074,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023871,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.11482,"completed_at":"2026-09-10T19:33:55.797Z","load_balancing_strategy":"primary","db_duration_s":0.07416,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.045371] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:55.801Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"d919e4a90300d836ea70b3b0","created_at":"2026-09-10T19:33:54.669Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a5424897018e5338d9ce138d7ddd4519","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-10T19:33:54.687Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-d919e4a90300d836ea70b3b0: start","job_status":"start","queue_duration_s":1.113771,"scheduling_latency_s":1.113826} gitlab # [ 276.169443] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:55.925Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"d919e4a90300d836ea70b3b0","created_at":"2026-09-10T19:33:54.669Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a5424897018e5338d9ce138d7ddd4519","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-10T19:33:54.687Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-d919e4a90300d836ea70b3b0: done: 0.123993 sec","job_status":"done","queue_duration_s":1.113771,"scheduling_latency_s":1.113826,"redis_calls":9,"redis_duration_s":0.005645,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001303,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003107,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001235,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.067,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.067,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037752,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.123993,"completed_at":"2026-09-10T19:33:55.925Z","load_balancing_strategy":"primary","db_duration_s":0.069128,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.173949] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:55.928Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"352d9cc873aeb9b4612aa3ff","created_at":"2026-09-10T19:33:54.708Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ebc51a8144c4c26c4e559e5b7e685bd3","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:54.726Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-352d9cc873aeb9b4612aa3ff: start","job_status":"start","queue_duration_s":1.202572,"scheduling_latency_s":1.202627} gitlab # [ 276.229486] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:55.985Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"352d9cc873aeb9b4612aa3ff","created_at":"2026-09-10T19:33:54.708Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ebc51a8144c4c26c4e559e5b7e685bd3","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:54.726Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-352d9cc873aeb9b4612aa3ff: done: 0.05689 sec","job_status":"done","queue_duration_s":1.202572,"scheduling_latency_s":1.202627,"redis_calls":28,"redis_duration_s":0.012487999999999999,"redis_read_bytes":36,"redis_write_bytes":10603,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001815,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":680,"redis_queues_calls":2,"redis_queues_duration_s":0.000582,"redis_queues_read_bytes":3,"redis_queues_write_bytes":8315,"redis_queues_metadata_calls":16,"redis_queues_metadata_duration_s":0.0069,"redis_queues_metadata_read_bytes":24,"redis_queues_metadata_write_bytes":1162,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003191,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":446,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034456,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000545321,"duration_s":0.05689,"completed_at":"2026-09-10T19:33:55.985Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002526,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.259566] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:55.987Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"b8aac554627851059ee480bf","created_at":"2026-09-10T19:33:54.746Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"93461b7e69dda31e3c311d2c5001e732","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:54.763Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-b8aac554627851059ee480bf: start","job_status":"start","queue_duration_s":1.224047,"scheduling_latency_s":1.224068} gitlab # [ 276.346751] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:55.999Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 276.348469] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.006Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"b8aac554627851059ee480bf","created_at":"2026-09-10T19:33:54.746Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"93461b7e69dda31e3c311d2c5001e732","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:54.763Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-b8aac554627851059ee480bf: done: 0.018951 sec","job_status":"done","queue_duration_s":1.224047,"scheduling_latency_s":1.224068,"redis_calls":9,"redis_duration_s":0.003229,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000432,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001961,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000836,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010333,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018951,"completed_at":"2026-09-10T19:33:56.006Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000626,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.354855] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.008Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"a0c6a504e9d93278ff998730","created_at":"2026-09-10T19:33:54.783Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eabd963ed5dca527afe092540b968d9f","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:54.881Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-a0c6a504e9d93278ff998730: start","job_status":"start","queue_duration_s":1.126734,"scheduling_latency_s":1.126756} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses gitlab # [ 276.595229] gitaly[931]: time="2026-09-10T19:33:56.351Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26CZZE7N0K0Y7FRYGYAGYN5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:34:26.149" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:56.350" grpc.time_ms=0.626 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 276.662845] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:56 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 122 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses, in 0.17 seconds) gitlab # [ 276.740101] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.496Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"a0c6a504e9d93278ff998730","created_at":"2026-09-10T19:33:54.783Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eabd963ed5dca527afe092540b968d9f","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:54.881Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-a0c6a504e9d93278ff998730: done: 0.487841 sec","job_status":"done","queue_duration_s":1.126734,"scheduling_latency_s":1.126756,"redis_calls":564,"redis_duration_s":0.171288,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00104,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001667,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.168581,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.359737,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.487841,"completed_at":"2026-09-10T19:33:56.496Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000796,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.742738] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.498Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"b84a496ac7b2f7d8489cbb1f","created_at":"2026-09-10T19:33:54.891Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0c277f5a10478a5640c932f15e623f80","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:54.904Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-b84a496ac7b2f7d8489cbb1f: start","job_status":"start","queue_duration_s":1.593962,"scheduling_latency_s":1.593991} gitlab # [ 276.842352] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.512Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 276.844105] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.518Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"b84a496ac7b2f7d8489cbb1f","created_at":"2026-09-10T19:33:54.891Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0c277f5a10478a5640c932f15e623f80","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:54.904Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-b84a496ac7b2f7d8489cbb1f: done: 0.020006 sec","job_status":"done","queue_duration_s":1.593962,"scheduling_latency_s":1.593991,"redis_calls":9,"redis_duration_s":0.003768,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000646,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001908,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001214,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01057,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020006,"completed_at":"2026-09-10T19:33:56.518Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001106,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.863581] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.520Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"f69bfce4f8a0cefbf76fa4a0","created_at":"2026-09-10T19:33:54.916Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"25a5d78b360d3f043f36342156ca1d0e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:54.930Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-f69bfce4f8a0cefbf76fa4a0: start","job_status":"start","queue_duration_s":1.589983,"scheduling_latency_s":1.590005} gitlab # [ 276.886150] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.531Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 276.955753] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.537Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"f69bfce4f8a0cefbf76fa4a0","created_at":"2026-09-10T19:33:54.916Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"25a5d78b360d3f043f36342156ca1d0e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:54.930Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-f69bfce4f8a0cefbf76fa4a0: done: 0.017724 sec","job_status":"done","queue_duration_s":1.589983,"scheduling_latency_s":1.590005,"redis_calls":9,"redis_duration_s":0.0032630000000000003,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000781,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001757,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000725,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00944,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017724,"completed_at":"2026-09-10T19:33:56.537Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000722,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.958393] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.539Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"07829c00c529828b5925554c","created_at":"2026-09-10T19:33:54.954Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1269b487c0a1a6e525b619c6f5dc9b45","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:54.967Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-07829c00c529828b5925554c: start","job_status":"start","queue_duration_s":1.571684,"scheduling_latency_s":1.571705} gitlab # [ 276.980801] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.560Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"07829c00c529828b5925554c","created_at":"2026-09-10T19:33:54.954Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1269b487c0a1a6e525b619c6f5dc9b45","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:54.967Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-07829c00c529828b5925554c: done: 0.021127 sec","job_status":"done","queue_duration_s":1.571684,"scheduling_latency_s":1.571705,"redis_calls":11,"redis_duration_s":0.004768,"redis_read_bytes":9,"redis_write_bytes":1459,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001804,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002163,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000801,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011403,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021127,"completed_at":"2026-09-10T19:33:56.560Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001449,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.046851] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.562Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"00522e48ee55ce407a214e66","created_at":"2026-09-10T19:33:54.993Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4836576e8ff7ad529a9277d09824bd66","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.017Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-00522e48ee55ce407a214e66: start","job_status":"start","queue_duration_s":1.545433,"scheduling_latency_s":1.545502} gitlab # [ 277.122771] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.591Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"00522e48ee55ce407a214e66","created_at":"2026-09-10T19:33:54.993Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4836576e8ff7ad529a9277d09824bd66","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.017Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-00522e48ee55ce407a214e66: done: 0.028638 sec","job_status":"done","queue_duration_s":1.545433,"scheduling_latency_s":1.545502,"redis_calls":12,"redis_duration_s":0.005383,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000824,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002543,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002016,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015799,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.007374401,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.028638,"completed_at":"2026-09-10T19:33:56.591Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003883,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 277.137314] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.593Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"abd2b4d78124b2fe86212836","created_at":"2026-09-10T19:33:55.050Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6c64f5df51d0ba4856b2f1264d17739c","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.071Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-abd2b4d78124b2fe86212836: start","job_status":"start","queue_duration_s":1.521352,"scheduling_latency_s":1.521373} gitlab # [ 277.159775] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.627Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"abd2b4d78124b2fe86212836","created_at":"2026-09-10T19:33:55.050Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6c64f5df51d0ba4856b2f1264d17739c","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.071Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-abd2b4d78124b2fe86212836: done: 0.034649 sec","job_status":"done","queue_duration_s":1.521352,"scheduling_latency_s":1.521373,"redis_calls":9,"redis_duration_s":0.003501,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000642,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001991,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000868,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02127,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034649,"completed_at":"2026-09-10T19:33:56.627Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002607,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.225118] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.630Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"21f3a1a26f6c6e2efa81d226","created_at":"2026-09-10T19:33:55.090Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1c22e5d0c4f5ce53cea4f87e807ed96e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.107Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-21f3a1a26f6c6e2efa81d226: start","job_status":"start","queue_duration_s":1.523484,"scheduling_latency_s":1.523534} gitlab # [ 277.246192] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.670Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"21f3a1a26f6c6e2efa81d226","created_at":"2026-09-10T19:33:55.090Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1c22e5d0c4f5ce53cea4f87e807ed96e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.107Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-21f3a1a26f6c6e2efa81d226: done: 0.040152 sec","job_status":"done","queue_duration_s":1.523484,"scheduling_latency_s":1.523534,"redis_calls":18,"redis_duration_s":0.004972,"redis_read_bytes":20,"redis_write_bytes":4698,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001432,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000314,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002406,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00082,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040152,"completed_at":"2026-09-10T19:33:56.670Z","load_balancing_strategy":"primary","db_duration_s":0.001898,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.312312] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.672Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"f6d7954d6129f5fc63aa11bd","created_at":"2026-09-10T19:33:55.125Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f19857f6f107f057bac3b84e2a60b24f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.142Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-f6d7954d6129f5fc63aa11bd: start","job_status":"start","queue_duration_s":1.529145,"scheduling_latency_s":1.529168} gitlab # [ 277.334351] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.682Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 277.339886] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.688Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"f6d7954d6129f5fc63aa11bd","created_at":"2026-09-10T19:33:55.125Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f19857f6f107f057bac3b84e2a60b24f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.142Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-f6d7954d6129f5fc63aa11bd: done: 0.016016 sec","job_status":"done","queue_duration_s":1.529145,"scheduling_latency_s":1.529168,"redis_calls":9,"redis_duration_s":0.002033,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000489,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001142,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000402,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009618,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016016,"completed_at":"2026-09-10T19:33:56.688Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000412,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.405482] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.689Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"bef31fa7f52984e598fd8acb","created_at":"2026-09-10T19:33:55.169Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f97bfcb7ab6dbf84df5f156c66d38e7c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.190Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-bef31fa7f52984e598fd8acb: start","job_status":"start","queue_duration_s":1.499067,"scheduling_latency_s":1.499088} gitlab # [ 277.426231] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.712Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"bef31fa7f52984e598fd8acb","created_at":"2026-09-10T19:33:55.169Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f97bfcb7ab6dbf84df5f156c66d38e7c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.190Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-bef31fa7f52984e598fd8acb: done: 0.023221 sec","job_status":"done","queue_duration_s":1.499067,"scheduling_latency_s":1.499088,"redis_calls":12,"redis_duration_s":0.0028009999999999997,"redis_read_bytes":11,"redis_write_bytes":1281,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000477,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0013,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":518,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001024,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":476,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013425,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000509562,"exclusive_lock_hold_duration_s":0.00741016,"duration_s":0.023221,"completed_at":"2026-09-10T19:33:56.712Z","load_balancing_strategy":"primary","db_duration_s":0.003807,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.542272] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.714Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"29ff19128ad8b1bea0639571","created_at":"2026-09-10T19:33:55.235Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"42b198574d5ac6a46824e8d0c71c321d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.296Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-29ff19128ad8b1bea0639571: start","job_status":"start","queue_duration_s":1.417014,"scheduling_latency_s":1.417034} gitlab # [ 277.543788] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.742Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"29ff19128ad8b1bea0639571","created_at":"2026-09-10T19:33:55.235Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"42b198574d5ac6a46824e8d0c71c321d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.296Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-29ff19128ad8b1bea0639571: done: 0.028337 sec","job_status":"done","queue_duration_s":1.417014,"scheduling_latency_s":1.417034,"redis_calls":11,"redis_duration_s":0.002643,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000441,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001254,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000948,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015497,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000464025,"exclusive_lock_hold_duration_s":0.007926705,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.028337,"completed_at":"2026-09-10T19:33:56.742Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007162,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.581282] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.743Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"310b66381dd5b198d35844c0","created_at":"2026-09-10T19:33:55.377Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ac44b9bfe26d7589ffe67fbedacc775","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.411Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-310b66381dd5b198d35844c0: start","job_status":"start","queue_duration_s":1.331774,"scheduling_latency_s":1.331796} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses gitlab # [ 277.602803] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.763Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"310b66381dd5b198d35844c0","created_at":"2026-09-10T19:33:55.377Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ac44b9bfe26d7589ffe67fbedacc775","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.411Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-310b66381dd5b198d35844c0: done: 0.02027 sec","job_status":"done","queue_duration_s":1.331774,"scheduling_latency_s":1.331796,"redis_calls":11,"redis_duration_s":0.002622,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000494,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001188,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00094,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012279,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00044894,"exclusive_lock_hold_duration_s":0.004056661,"duration_s":0.02027,"completed_at":"2026-09-10T19:33:56.763Z","load_balancing_strategy":"primary","db_duration_s":0.002099,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses, in 0.08 seconds) gitlab # [ 277.689299] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.765Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"6e2276682a5419e7adccfaa3","created_at":"2026-09-10T19:33:55.420Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"769a8894c6a1eb0950a19e5afd673064","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.431Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-6e2276682a5419e7adccfaa3: start","job_status":"start","queue_duration_s":1.333913,"scheduling_latency_s":1.333932} gitlab # [ 277.691632] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.783Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"6e2276682a5419e7adccfaa3","created_at":"2026-09-10T19:33:55.420Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"769a8894c6a1eb0950a19e5afd673064","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.431Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-6e2276682a5419e7adccfaa3: done: 0.018085 sec","job_status":"done","queue_duration_s":1.333913,"scheduling_latency_s":1.333932,"redis_calls":11,"redis_duration_s":0.002636,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000492,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001161,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000983,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01082,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00047101,"exclusive_lock_hold_duration_s":0.001784584,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.018085,"completed_at":"2026-09-10T19:33:56.783Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001108,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.768112] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.784Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"39fdae9786bdb29437a3b082","created_at":"2026-09-10T19:33:55.458Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f15b3d2657bc3c8429a0a3b34356110d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-10T19:33:55.478Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-39fdae9786bdb29437a3b082: start","job_status":"start","queue_duration_s":1.305725,"scheduling_latency_s":1.305748} gitlab # [ 277.776709] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.794Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 277.782408] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.800Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"39fdae9786bdb29437a3b082","created_at":"2026-09-10T19:33:55.458Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f15b3d2657bc3c8429a0a3b34356110d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-10T19:33:55.478Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-39fdae9786bdb29437a3b082: done: 0.015628 sec","job_status":"done","queue_duration_s":1.305725,"scheduling_latency_s":1.305748,"redis_calls":9,"redis_duration_s":0.001954,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000449,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001092,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000413,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009547,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015628,"completed_at":"2026-09-10T19:33:56.800Z","load_balancing_strategy":"primary","db_duration_s":0.000415,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.862639] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.801Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"4147a4864cd0e9c08244998e","created_at":"2026-09-10T19:33:55.507Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"792aa0d8a6bb106a30c418f5ad8b4069","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.526Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-4147a4864cd0e9c08244998e: start","job_status":"start","queue_duration_s":1.274952,"scheduling_latency_s":1.274973} gitlab # [ 277.869581] gitaly[931]: time="2026-09-10T19:33:57.492Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26D00JH47TRS38M01D3EA95 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:34:27.290" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:57.491" grpc.time_ms=0.85 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 277.955428] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:57 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 50 gitlab # [ 277.956359] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.817Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"4147a4864cd0e9c08244998e","created_at":"2026-09-10T19:33:55.507Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"792aa0d8a6bb106a30c418f5ad8b4069","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.526Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-4147a4864cd0e9c08244998e: done: 0.016088 sec","job_status":"done","queue_duration_s":1.274952,"scheduling_latency_s":1.274973,"redis_calls":9,"redis_duration_s":0.001966,"redis_read_bytes":7,"redis_write_bytes":1119,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000441,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001069,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000456,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009942,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016088,"completed_at":"2026-09-10T19:33:56.817Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000334,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.959161] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.819Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"20ae7213f1acb615540ab606","created_at":"2026-09-10T19:33:55.545Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.560Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-20ae7213f1acb615540ab606: start","job_status":"start","queue_duration_s":1.258455,"scheduling_latency_s":1.258477} gitlab # [ 278.046579] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.928Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"20ae7213f1acb615540ab606","created_at":"2026-09-10T19:33:55.545Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.560Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-20ae7213f1acb615540ab606: done: 0.109311 sec","job_status":"done","queue_duration_s":1.258455,"scheduling_latency_s":1.258477,"redis_calls":70,"redis_duration_s":0.0278,"redis_read_bytes":118,"redis_write_bytes":42530,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.003382,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001165,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.02217,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001083,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.076185,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.109311,"completed_at":"2026-09-10T19:33:56.928Z","load_balancing_strategy":"primary","db_duration_s":0.002567,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.048404] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.931Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"d66dbde6144d7ea2c614fe10","created_at":"2026-09-10T19:33:55.591Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f896d07746981b4b17d27c4be1e9cd3","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.608Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-d66dbde6144d7ea2c614fe10: start","job_status":"start","queue_duration_s":1.322552,"scheduling_latency_s":1.322607} gitlab # [ 278.068969] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.967Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"d66dbde6144d7ea2c614fe10","created_at":"2026-09-10T19:33:55.591Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f896d07746981b4b17d27c4be1e9cd3","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.608Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-d66dbde6144d7ea2c614fe10: done: 0.036362 sec","job_status":"done","queue_duration_s":1.322552,"scheduling_latency_s":1.322607,"redis_calls":9,"redis_duration_s":0.002735,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000595,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001554,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000586,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019736,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.036362,"completed_at":"2026-09-10T19:33:56.967Z","load_balancing_strategy":"primary","db_duration_s":0.006161,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.137370] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.968Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"374e2c9981f0dfb4cef3e613","created_at":"2026-09-10T19:33:55.959Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-44b7ebcc5e494250","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"ebc51a8144c4c26c4e559e5b7e685bd3","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.976Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-374e2c9981f0dfb4cef3e613: start","job_status":"start","queue_duration_s":0.991962,"scheduling_latency_s":0.991985} gitlab # [ 278.155429] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.993Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"374e2c9981f0dfb4cef3e613","created_at":"2026-09-10T19:33:55.959Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-44b7ebcc5e494250","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"ebc51a8144c4c26c4e559e5b7e685bd3","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.976Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-374e2c9981f0dfb4cef3e613: done: 0.024484 sec","job_status":"done","queue_duration_s":0.991962,"scheduling_latency_s":0.991985,"redis_calls":16,"redis_duration_s":0.003699,"redis_read_bytes":14,"redis_write_bytes":1526,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000427,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001305,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":423,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001967,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":802,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015193,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000344736,"duration_s":0.024484,"completed_at":"2026-09-10T19:33:56.993Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00139,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.229827] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:56.995Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"b6092dfc532621d1a5102d64","created_at":"2026-09-10T19:33:55.959Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-44b7ebcc5e494250","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"ebc51a8144c4c26c4e559e5b7e685bd3","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.976Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-b6092dfc532621d1a5102d64: start","job_status":"start","queue_duration_s":1.018132,"scheduling_latency_s":1.018153} gitlab # [ 278.303282] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.010Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"b6092dfc532621d1a5102d64","created_at":"2026-09-10T19:33:55.959Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-44b7ebcc5e494250","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"ebc51a8144c4c26c4e559e5b7e685bd3","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.976Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-b6092dfc532621d1a5102d64: done: 0.01589 sec","job_status":"done","queue_duration_s":1.018132,"scheduling_latency_s":1.018153,"redis_calls":13,"redis_duration_s":0.00322,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001072,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002148,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010427,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01589,"completed_at":"2026-09-10T19:33:57.010Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000497,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.305146] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.012Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"de829348a1b31ce874d96e8e","created_at":"2026-09-10T19:33:55.959Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-44b7ebcc5e494250","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"ebc51a8144c4c26c4e559e5b7e685bd3","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.976Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-de829348a1b31ce874d96e8e: start","job_status":"start","queue_duration_s":1.035275,"scheduling_latency_s":1.035296} gitlab # [ 278.375222] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.028Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"de829348a1b31ce874d96e8e","created_at":"2026-09-10T19:33:55.959Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-44b7ebcc5e494250","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"ebc51a8144c4c26c4e559e5b7e685bd3","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.976Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-de829348a1b31ce874d96e8e: done: 0.015835 sec","job_status":"done","queue_duration_s":1.035275,"scheduling_latency_s":1.035296,"redis_calls":13,"redis_duration_s":0.002805,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00083,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001975,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010065,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015835,"completed_at":"2026-09-10T19:33:57.027Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00048,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.382081] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.029Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"788f198211182803ea6a6654","created_at":"2026-09-10T19:33:55.959Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-44b7ebcc5e494250","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"ebc51a8144c4c26c4e559e5b7e685bd3","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.976Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-788f198211182803ea6a6654: start","job_status":"start","queue_duration_s":1.052356,"scheduling_latency_s":1.052377} gitlab # [ 278.450361] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.044Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"788f198211182803ea6a6654","created_at":"2026-09-10T19:33:55.959Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-44b7ebcc5e494250","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"ebc51a8144c4c26c4e559e5b7e685bd3","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.976Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-788f198211182803ea6a6654: done: 0.015484 sec","job_status":"done","queue_duration_s":1.052356,"scheduling_latency_s":1.052377,"redis_calls":13,"redis_duration_s":0.002691,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00084,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001851,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010044,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015484,"completed_at":"2026-09-10T19:33:57.044Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000478,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.464260] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.046Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"0ba24f5d338e86a14d3c8f1f","created_at":"2026-09-10T19:33:55.959Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-44b7ebcc5e494250","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"ebc51a8144c4c26c4e559e5b7e685bd3","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.976Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-0ba24f5d338e86a14d3c8f1f: start","job_status":"start","queue_duration_s":1.069408,"scheduling_latency_s":1.069429} gitlab # [ 278.535555] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.061Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"0ba24f5d338e86a14d3c8f1f","created_at":"2026-09-10T19:33:55.959Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-44b7ebcc5e494250","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"ebc51a8144c4c26c4e559e5b7e685bd3","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.976Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-0ba24f5d338e86a14d3c8f1f: done: 0.015334 sec","job_status":"done","queue_duration_s":1.069408,"scheduling_latency_s":1.069429,"redis_calls":13,"redis_duration_s":0.003023,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001221,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001802,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010243,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015334,"completed_at":"2026-09-10T19:33:57.061Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000448,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.546103] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.063Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"c61528b54c065d1a47f0eb85","created_at":"2026-09-10T19:33:55.959Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-44b7ebcc5e494250","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"ebc51a8144c4c26c4e559e5b7e685bd3","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.976Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-c61528b54c065d1a47f0eb85: start","job_status":"start","queue_duration_s":1.086052,"scheduling_latency_s":1.086074} gitlab # [ 278.567535] nix-daemon[2886]: accepted connection from pid 3809, user nixbot gitlab # [ 278.568755] nix-daemon[2886]: accepted connection from pid 3771, user nixbot gitlab # [ 278.569916] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.078Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"c61528b54c065d1a47f0eb85","created_at":"2026-09-10T19:33:55.959Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-44b7ebcc5e494250","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"ebc51a8144c4c26c4e559e5b7e685bd3","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.976Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-c61528b54c065d1a47f0eb85: done: 0.015783 sec","job_status":"done","queue_duration_s":1.086052,"scheduling_latency_s":1.086074,"redis_calls":13,"redis_duration_s":0.002697,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000829,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001868,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015783,"completed_at":"2026-09-10T19:33:57.078Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000525,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.637485] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.080Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"3fc52b39ab79a6d7a8296efa","created_at":"2026-09-10T19:33:55.959Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-44b7ebcc5e494250","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"ebc51a8144c4c26c4e559e5b7e685bd3","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.976Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-3fc52b39ab79a6d7a8296efa: start","job_status":"start","queue_duration_s":1.103118,"scheduling_latency_s":1.103141} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses gitlab # [ 278.651416] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.095Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"3fc52b39ab79a6d7a8296efa","created_at":"2026-09-10T19:33:55.959Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-44b7ebcc5e494250","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"ebc51a8144c4c26c4e559e5b7e685bd3","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.976Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-3fc52b39ab79a6d7a8296efa: done: 0.015633 sec","job_status":"done","queue_duration_s":1.103118,"scheduling_latency_s":1.103141,"redis_calls":13,"redis_duration_s":0.002673,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000893,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00178,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015633,"completed_at":"2026-09-10T19:33:57.095Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000466,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.719933] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.097Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"a75b577056658083e411c22f","created_at":"2026-09-10T19:33:55.959Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-44b7ebcc5e494250","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"ebc51a8144c4c26c4e559e5b7e685bd3","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.976Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-a75b577056658083e411c22f: start","job_status":"start","queue_duration_s":1.12018,"scheduling_latency_s":1.1202} gitlab # [ 278.796388] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.112Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"a75b577056658083e411c22f","created_at":"2026-09-10T19:33:55.959Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-44b7ebcc5e494250","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"ebc51a8144c4c26c4e559e5b7e685bd3","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.976Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-a75b577056658083e411c22f: done: 0.015311 sec","job_status":"done","queue_duration_s":1.12018,"scheduling_latency_s":1.1202,"redis_calls":13,"redis_duration_s":0.002896,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00087,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002026,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010042,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015311,"completed_at":"2026-09-10T19:33:57.112Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000497,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.798186] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.113Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"6b1abb0a3c3edb6bb98616e4","created_at":"2026-09-10T19:33:55.959Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-44b7ebcc5e494250","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"ebc51a8144c4c26c4e559e5b7e685bd3","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.976Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-6b1abb0a3c3edb6bb98616e4: start","job_status":"start","queue_duration_s":1.136787,"scheduling_latency_s":1.136808} gitlab # [ 278.875933] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.128Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"6b1abb0a3c3edb6bb98616e4","created_at":"2026-09-10T19:33:55.959Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-44b7ebcc5e494250","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"ebc51a8144c4c26c4e559e5b7e685bd3","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.976Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-6b1abb0a3c3edb6bb98616e4: done: 0.015218 sec","job_status":"done","queue_duration_s":1.136787,"scheduling_latency_s":1.136808,"redis_calls":13,"redis_duration_s":0.0027010000000000003,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001793,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009827,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015218,"completed_at":"2026-09-10T19:33:57.128Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000459,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses, in 0.19 seconds) gitlab # [ 278.877373] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.130Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"c4cda76a86863b3dc0172a05","created_at":"2026-09-10T19:33:55.959Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-44b7ebcc5e494250","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"ebc51a8144c4c26c4e559e5b7e685bd3","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.976Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-c4cda76a86863b3dc0172a05: start","job_status":"start","queue_duration_s":1.153209,"scheduling_latency_s":1.15323} gitlab # [ 278.894115] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.145Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"c4cda76a86863b3dc0172a05","created_at":"2026-09-10T19:33:55.959Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-44b7ebcc5e494250","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"ebc51a8144c4c26c4e559e5b7e685bd3","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:55.976Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-c4cda76a86863b3dc0172a05: done: 0.014953 sec","job_status":"done","queue_duration_s":1.153209,"scheduling_latency_s":1.15323,"redis_calls":13,"redis_duration_s":0.002629,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000837,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001792,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009778,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014953,"completed_at":"2026-09-10T19:33:57.145Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000491,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.962273] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.147Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"bf1865d996ed4eddecba9935","created_at":"2026-09-10T19:33:56.653Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-f0c00198de7344d4","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1c22e5d0c4f5ce53cea4f87e807ed96e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.664Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-bf1865d996ed4eddecba9935: start","job_status":"start","queue_duration_s":0.482496,"scheduling_latency_s":0.482518} gitlab # [ 279.031282] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.161Z","message":"No existing merge request to be cleaned up."} gitlab # [ 279.032428] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.171Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"bf1865d996ed4eddecba9935","created_at":"2026-09-10T19:33:56.653Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-f0c00198de7344d4","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1c22e5d0c4f5ce53cea4f87e807ed96e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.664Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-bf1865d996ed4eddecba9935: done: 0.024296 sec","job_status":"done","queue_duration_s":0.482496,"scheduling_latency_s":0.482518,"redis_calls":12,"redis_duration_s":0.00247,"redis_read_bytes":10,"redis_write_bytes":1323,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000387,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000871,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001212,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015766,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024296,"completed_at":"2026-09-10T19:33:57.171Z","load_balancing_strategy":"primary","db_duration_s":0.002801,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.036399] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.173Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ab14657f1d31119978a1df17","created_at":"2026-09-10T19:33:56.653Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-f0c00198de7344d4","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1c22e5d0c4f5ce53cea4f87e807ed96e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.664Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ab14657f1d31119978a1df17: start","job_status":"start","queue_duration_s":0.508126,"scheduling_latency_s":0.508156} gitlab # [ 279.055989] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.183Z","message":"No existing merge request to be cleaned up."} gitlab # [ 279.058704] gitaly[931]: time="2026-09-10T19:33:58.674Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26D01MC9XVRPYT23TV6CNGQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:34:28.373" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:58.673" grpc.time_ms=0.59 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 279.078552] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:58 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 158 gitlab # [ 279.082393] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.189Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ab14657f1d31119978a1df17","created_at":"2026-09-10T19:33:56.653Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-f0c00198de7344d4","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1c22e5d0c4f5ce53cea4f87e807ed96e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.664Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ab14657f1d31119978a1df17: done: 0.016491 sec","job_status":"done","queue_duration_s":0.508126,"scheduling_latency_s":0.508156,"redis_calls":10,"redis_duration_s":0.002092,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000843,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001249,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011184,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016491,"completed_at":"2026-09-10T19:33:57.189Z","load_balancing_strategy":"primary","db_duration_s":0.001439,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.143078] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.191Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"62792b15bfcdbe8389e3627e","created_at":"2026-09-10T19:33:56.653Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-f0c00198de7344d4","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1c22e5d0c4f5ce53cea4f87e807ed96e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.664Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-62792b15bfcdbe8389e3627e: start","job_status":"start","queue_duration_s":0.526348,"scheduling_latency_s":0.52637} gitlab # [ 279.161919] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.201Z","message":"No existing merge request to be cleaned up."} gitlab # [ 279.164723] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.207Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"62792b15bfcdbe8389e3627e","created_at":"2026-09-10T19:33:56.653Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-f0c00198de7344d4","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1c22e5d0c4f5ce53cea4f87e807ed96e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.664Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-62792b15bfcdbe8389e3627e: done: 0.016054 sec","job_status":"done","queue_duration_s":0.526348,"scheduling_latency_s":0.52637,"redis_calls":10,"redis_duration_s":0.002138,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001221,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010806,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016054,"completed_at":"2026-09-10T19:33:57.207Z","load_balancing_strategy":"primary","db_duration_s":0.00143,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.224776] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.209Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ea142a5eac3cedc96b83b017","created_at":"2026-09-10T19:33:56.653Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-f0c00198de7344d4","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1c22e5d0c4f5ce53cea4f87e807ed96e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.664Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ea142a5eac3cedc96b83b017: start","job_status":"start","queue_duration_s":0.54408,"scheduling_latency_s":0.544101} gitlab # [ 279.244090] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.219Z","message":"No existing merge request to be cleaned up."} gitlab # [ 279.246325] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.225Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ea142a5eac3cedc96b83b017","created_at":"2026-09-10T19:33:56.653Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-f0c00198de7344d4","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1c22e5d0c4f5ce53cea4f87e807ed96e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.664Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ea142a5eac3cedc96b83b017: done: 0.016381 sec","job_status":"done","queue_duration_s":0.54408,"scheduling_latency_s":0.544101,"redis_calls":10,"redis_duration_s":0.002149,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000831,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001318,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010784,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016381,"completed_at":"2026-09-10T19:33:57.225Z","load_balancing_strategy":"primary","db_duration_s":0.001446,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.306322] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.226Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"88c1469f164d5886295d13cd","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-88c1469f164d5886295d13cd: start","job_status":"start","queue_duration_s":0.313441,"scheduling_latency_s":0.313462} gitlab # [ 279.326083] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.247Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"88c1469f164d5886295d13cd","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-88c1469f164d5886295d13cd: done: 0.021046 sec","job_status":"done","queue_duration_s":0.313441,"scheduling_latency_s":0.313462,"redis_calls":15,"redis_duration_s":0.003179,"redis_read_bytes":12,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000406,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000813,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00196,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013261,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021046,"completed_at":"2026-09-10T19:33:57.247Z","load_balancing_strategy":"primary","db_duration_s":0.000344,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.389897] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.249Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a8de7f642567d1b3cf3af74","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a8de7f642567d1b3cf3af74: start","job_status":"start","queue_duration_s":0.335783,"scheduling_latency_s":0.335806} gitlab # [ 279.409324] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.264Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a8de7f642567d1b3cf3af74","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a8de7f642567d1b3cf3af74: done: 0.015432 sec","job_status":"done","queue_duration_s":0.335783,"scheduling_latency_s":0.335806,"redis_calls":13,"redis_duration_s":0.002998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000875,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002123,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010283,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015432,"completed_at":"2026-09-10T19:33:57.264Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.470120] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.265Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d6337893b9a0d9da0f51d1c","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d6337893b9a0d9da0f51d1c: start","job_status":"start","queue_duration_s":0.352544,"scheduling_latency_s":0.352563} gitlab # [ 279.489442] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.282Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d6337893b9a0d9da0f51d1c","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d6337893b9a0d9da0f51d1c: done: 0.016715 sec","job_status":"done","queue_duration_s":0.352544,"scheduling_latency_s":0.352563,"redis_calls":13,"redis_duration_s":0.004114,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002017,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002097,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010074,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016715,"completed_at":"2026-09-10T19:33:57.282Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.550785] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.284Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"247b352e3b4ddbe33eb04a9a","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-247b352e3b4ddbe33eb04a9a: start","job_status":"start","queue_duration_s":0.37058,"scheduling_latency_s":0.370601} gitlab # [ 279.570134] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.298Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"247b352e3b4ddbe33eb04a9a","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-247b352e3b4ddbe33eb04a9a: done: 0.014916 sec","job_status":"done","queue_duration_s":0.37058,"scheduling_latency_s":0.370601,"redis_calls":13,"redis_duration_s":0.002709,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00087,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001839,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01002,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014916,"completed_at":"2026-09-10T19:33:57.298Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.666187] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.300Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3dbc717f4120a566ccbc468c","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3dbc717f4120a566ccbc468c: start","job_status":"start","queue_duration_s":0.386765,"scheduling_latency_s":0.386786} gitlab # [ 279.667347] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.315Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3dbc717f4120a566ccbc468c","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3dbc717f4120a566ccbc468c: done: 0.014907 sec","job_status":"done","queue_duration_s":0.386765,"scheduling_latency_s":0.386786,"redis_calls":13,"redis_duration_s":0.0028450000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000805,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00204,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009974,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014907,"completed_at":"2026-09-10T19:33:57.315Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.709461] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.316Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f3d078875e809022af545b8e","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f3d078875e809022af545b8e: start","job_status":"start","queue_duration_s":0.402961,"scheduling_latency_s":0.402982} gitlab # [ 279.729105] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.331Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f3d078875e809022af545b8e","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f3d078875e809022af545b8e: done: 0.014831 sec","job_status":"done","queue_duration_s":0.402961,"scheduling_latency_s":0.402982,"redis_calls":13,"redis_duration_s":0.002638,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000814,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001824,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00998,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014831,"completed_at":"2026-09-10T19:33:57.331Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.806235] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.332Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6fa4513d78bb8c310396c59f","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6fa4513d78bb8c310396c59f: start","job_status":"start","queue_duration_s":0.419058,"scheduling_latency_s":0.41908} gitlab # [ 279.807830] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.347Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6fa4513d78bb8c310396c59f","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6fa4513d78bb8c310396c59f: done: 0.014967 sec","job_status":"done","queue_duration_s":0.419058,"scheduling_latency_s":0.41908,"redis_calls":13,"redis_duration_s":0.002621,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000845,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001776,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010024,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014967,"completed_at":"2026-09-10T19:33:57.347Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses gitlab # [ 279.868476] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.348Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e65e4aa9ad2727ff0852fffc","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e65e4aa9ad2727ff0852fffc: start","job_status":"start","queue_duration_s":0.435337,"scheduling_latency_s":0.435357} gitlab # [ 279.889188] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.363Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e65e4aa9ad2727ff0852fffc","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e65e4aa9ad2727ff0852fffc: done: 0.014825 sec","job_status":"done","queue_duration_s":0.435337,"scheduling_latency_s":0.435357,"redis_calls":13,"redis_duration_s":0.00268,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000819,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001861,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009986,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014825,"completed_at":"2026-09-10T19:33:57.363Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses, in 0.08 seconds) gitlab # [ 279.949864] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.364Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8a9ed8e9607df79598fd610","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8a9ed8e9607df79598fd610: start","job_status":"start","queue_duration_s":0.451416,"scheduling_latency_s":0.451437} gitlab # [ 279.969224] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.379Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8a9ed8e9607df79598fd610","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8a9ed8e9607df79598fd610: done: 0.014761 sec","job_status":"done","queue_duration_s":0.451416,"scheduling_latency_s":0.451437,"redis_calls":13,"redis_duration_s":0.0027,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000834,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001866,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009933,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014761,"completed_at":"2026-09-10T19:33:57.379Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.030535] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.380Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af22c2cdfb91be378c722bf6","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af22c2cdfb91be378c722bf6: start","job_status":"start","queue_duration_s":0.467521,"scheduling_latency_s":0.467541} gitlab # [ 280.050224] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.395Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af22c2cdfb91be378c722bf6","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af22c2cdfb91be378c722bf6: done: 0.014837 sec","job_status":"done","queue_duration_s":0.467521,"scheduling_latency_s":0.467541,"redis_calls":13,"redis_duration_s":0.002684,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000885,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001799,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00997,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014837,"completed_at":"2026-09-10T19:33:57.395Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.133243] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14eef9d049c941fa1ef7b69c","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14eef9d049c941fa1ef7b69c: start","job_status":"start","queue_duration_s":0.483657,"scheduling_latency_s":0.483677} gitlab # [ 280.135302] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:33:59 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 56 gitlab # [ 280.136500] gitaly[931]: time="2026-09-10T19:33:59.767Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26D02SF1HA0VP9FJ3MYJ4Q3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:34:29.465" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:33:59.765" grpc.time_ms=0.824 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 280.205289] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.411Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14eef9d049c941fa1ef7b69c","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14eef9d049c941fa1ef7b69c: done: 0.014911 sec","job_status":"done","queue_duration_s":0.483657,"scheduling_latency_s":0.483677,"redis_calls":13,"redis_duration_s":0.002767,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001833,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010021,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014911,"completed_at":"2026-09-10T19:33:57.411Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.214927] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.413Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b14524894858ca9a82edecb1","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b14524894858ca9a82edecb1: start","job_status":"start","queue_duration_s":0.499882,"scheduling_latency_s":0.499903} gitlab # [ 280.284425] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.429Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b14524894858ca9a82edecb1","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b14524894858ca9a82edecb1: done: 0.015812 sec","job_status":"done","queue_duration_s":0.499882,"scheduling_latency_s":0.499903,"redis_calls":13,"redis_duration_s":0.003606,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001217,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002389,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01001,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015812,"completed_at":"2026-09-10T19:33:57.429Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.294858] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.430Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ac581b71ddf2c2055a55e3eb","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ac581b71ddf2c2055a55e3eb: start","job_status":"start","queue_duration_s":0.517417,"scheduling_latency_s":0.517438} gitlab # [ 280.367297] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.450Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ac581b71ddf2c2055a55e3eb","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ac581b71ddf2c2055a55e3eb: done: 0.019568 sec","job_status":"done","queue_duration_s":0.517417,"scheduling_latency_s":0.517438,"redis_calls":13,"redis_duration_s":0.007031,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001876,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005155,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01028,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019568,"completed_at":"2026-09-10T19:33:57.450Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.374944] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.452Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03a66918a0c0347c1427ec12","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03a66918a0c0347c1427ec12: start","job_status":"start","queue_duration_s":0.538966,"scheduling_latency_s":0.538988} gitlab # [ 280.394489] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.468Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03a66918a0c0347c1427ec12","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03a66918a0c0347c1427ec12: done: 0.016567 sec","job_status":"done","queue_duration_s":0.538966,"scheduling_latency_s":0.538988,"redis_calls":13,"redis_duration_s":0.003567,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002063,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010252,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016567,"completed_at":"2026-09-10T19:33:57.468Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.481945] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.470Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"06e8d38c32725b701224398a","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-06e8d38c32725b701224398a: start","job_status":"start","queue_duration_s":0.556904,"scheduling_latency_s":0.556925} gitlab # [ 280.483069] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.485Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"06e8d38c32725b701224398a","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-06e8d38c32725b701224398a: done: 0.015246 sec","job_status":"done","queue_duration_s":0.556904,"scheduling_latency_s":0.556925,"redis_calls":13,"redis_duration_s":0.0026579999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000846,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001812,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010257,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015246,"completed_at":"2026-09-10T19:33:57.485Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.558935] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.486Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a1292f33616e33a791fa8ce","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a1292f33616e33a791fa8ce: start","job_status":"start","queue_duration_s":0.57347,"scheduling_latency_s":0.573491} gitlab # [ 280.560182] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.502Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a1292f33616e33a791fa8ce","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a1292f33616e33a791fa8ce: done: 0.015478 sec","job_status":"done","queue_duration_s":0.57347,"scheduling_latency_s":0.573491,"redis_calls":13,"redis_duration_s":0.0029,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00105,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00185,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010338,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015478,"completed_at":"2026-09-10T19:33:57.502Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.634226] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.503Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d552040c7f9c2eee548a6f2a","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d552040c7f9c2eee548a6f2a: start","job_status":"start","queue_duration_s":0.590244,"scheduling_latency_s":0.590303} gitlab # [ 280.636294] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.519Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d552040c7f9c2eee548a6f2a","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d552040c7f9c2eee548a6f2a: done: 0.015541 sec","job_status":"done","queue_duration_s":0.590244,"scheduling_latency_s":0.590303,"redis_calls":13,"redis_duration_s":0.0030629999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001163,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0019,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010258,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015541,"completed_at":"2026-09-10T19:33:57.519Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.755332] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.520Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19907d9a326ef407bdc19e22","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19907d9a326ef407bdc19e22: start","job_status":"start","queue_duration_s":0.607056,"scheduling_latency_s":0.607076} gitlab # [ 280.756482] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.535Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19907d9a326ef407bdc19e22","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19907d9a326ef407bdc19e22: done: 0.015211 sec","job_status":"done","queue_duration_s":0.607056,"scheduling_latency_s":0.607076,"redis_calls":13,"redis_duration_s":0.002889,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00197,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009726,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015211,"completed_at":"2026-09-10T19:33:57.535Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.842460] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.537Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9361438d1f7b8136fee29579","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9361438d1f7b8136fee29579: start","job_status":"start","queue_duration_s":0.624079,"scheduling_latency_s":0.624101} gitlab # [ 280.843832] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.552Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9361438d1f7b8136fee29579","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9361438d1f7b8136fee29579: done: 0.015207 sec","job_status":"done","queue_duration_s":0.624079,"scheduling_latency_s":0.624101,"redis_calls":13,"redis_duration_s":0.003019,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001023,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001996,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009761,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015207,"completed_at":"2026-09-10T19:33:57.552Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.922349] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.554Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4e858561d1a4854e3c36872","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4e858561d1a4854e3c36872: start","job_status":"start","queue_duration_s":0.640592,"scheduling_latency_s":0.640614} gitlab # [ 280.923480] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.568Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4e858561d1a4854e3c36872","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4e858561d1a4854e3c36872: done: 0.014598 sec","job_status":"done","queue_duration_s":0.640592,"scheduling_latency_s":0.640614,"redis_calls":13,"redis_duration_s":0.002582,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000815,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001767,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009979,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014598,"completed_at":"2026-09-10T19:33:57.568Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.999439] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.569Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"355263022ba372102d86bfb4","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-355263022ba372102d86bfb4: start","job_status":"start","queue_duration_s":0.656501,"scheduling_latency_s":0.656522} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses gitlab # [ 281.000802] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.584Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"355263022ba372102d86bfb4","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-355263022ba372102d86bfb4: done: 0.014545 sec","job_status":"done","queue_duration_s":0.656501,"scheduling_latency_s":0.656522,"redis_calls":13,"redis_duration_s":0.00263,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000816,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001814,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009898,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014545,"completed_at":"2026-09-10T19:33:57.584Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.015277] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.585Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5172822d26d9f22a6f9e751","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5172822d26d9f22a6f9e751: start","job_status":"start","queue_duration_s":0.672343,"scheduling_latency_s":0.672364} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses, in 0.09 seconds) gitlab # [ 281.034914] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.600Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5172822d26d9f22a6f9e751","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5172822d26d9f22a6f9e751: done: 0.014644 sec","job_status":"done","queue_duration_s":0.672343,"scheduling_latency_s":0.672364,"redis_calls":13,"redis_duration_s":0.002672,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000827,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001845,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009908,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014644,"completed_at":"2026-09-10T19:33:57.600Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.110227] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.601Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea1b2ee916c5d71fd19a298b","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea1b2ee916c5d71fd19a298b: start","job_status":"start","queue_duration_s":0.688227,"scheduling_latency_s":0.688247} gitlab # [ 281.112911] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.616Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea1b2ee916c5d71fd19a298b","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea1b2ee916c5d71fd19a298b: done: 0.014807 sec","job_status":"done","queue_duration_s":0.688227,"scheduling_latency_s":0.688247,"redis_calls":13,"redis_duration_s":0.002811,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000807,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002004,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00994,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014807,"completed_at":"2026-09-10T19:33:57.616Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.185795] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.617Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34f66ab9010191e068858dcd","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34f66ab9010191e068858dcd: start","job_status":"start","queue_duration_s":0.704357,"scheduling_latency_s":0.704378} gitlab # [ 281.191447] gitaly[931]: time="2026-09-10T19:34:00.861Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26D03VGQBVEYAXAKK3VCCMT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:34:30.560" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:34:00.860" grpc.time_ms=0.561 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 281.211452] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:34:00 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 61 gitlab # [ 281.215263] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.632Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34f66ab9010191e068858dcd","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34f66ab9010191e068858dcd: done: 0.014644 sec","job_status":"done","queue_duration_s":0.704357,"scheduling_latency_s":0.704378,"redis_calls":13,"redis_duration_s":0.002684,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000847,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001837,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009975,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014644,"completed_at":"2026-09-10T19:33:57.632Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.276155] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.633Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f7568096c6d4348bbf91e40","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f7568096c6d4348bbf91e40: start","job_status":"start","queue_duration_s":0.720248,"scheduling_latency_s":0.720304} gitlab # [ 281.295247] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.648Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f7568096c6d4348bbf91e40","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f7568096c6d4348bbf91e40: done: 0.014708 sec","job_status":"done","queue_duration_s":0.720248,"scheduling_latency_s":0.720304,"redis_calls":13,"redis_duration_s":0.002606,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000852,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001754,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009966,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014708,"completed_at":"2026-09-10T19:33:57.648Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.356542] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.649Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"413ea640c1fe26745c1ed610","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-413ea640c1fe26745c1ed610: start","job_status":"start","queue_duration_s":0.736192,"scheduling_latency_s":0.736214} gitlab # [ 281.376045] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.664Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"413ea640c1fe26745c1ed610","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-413ea640c1fe26745c1ed610: done: 0.014644 sec","job_status":"done","queue_duration_s":0.736192,"scheduling_latency_s":0.736214,"redis_calls":13,"redis_duration_s":0.002533,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000777,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001756,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009983,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014644,"completed_at":"2026-09-10T19:33:57.664Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.436829] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.665Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"87dc17d2906eb531f6806a35","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-87dc17d2906eb531f6806a35: start","job_status":"start","queue_duration_s":0.75209,"scheduling_latency_s":0.752111} gitlab # [ 281.456174] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.680Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"87dc17d2906eb531f6806a35","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-87dc17d2906eb531f6806a35: done: 0.0146 sec","job_status":"done","queue_duration_s":0.75209,"scheduling_latency_s":0.752111,"redis_calls":13,"redis_duration_s":0.00257,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000783,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001787,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00995,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0146,"completed_at":"2026-09-10T19:33:57.680Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.516443] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.681Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d08ba444164725d75e3bd842","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d08ba444164725d75e3bd842: start","job_status":"start","queue_duration_s":0.767926,"scheduling_latency_s":0.767945} gitlab # [ 281.535783] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.695Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d08ba444164725d75e3bd842","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d08ba444164725d75e3bd842: done: 0.014535 sec","job_status":"done","queue_duration_s":0.767926,"scheduling_latency_s":0.767945,"redis_calls":13,"redis_duration_s":0.0025900000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000815,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001775,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009893,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014535,"completed_at":"2026-09-10T19:33:57.695Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.597170] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.697Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b6115104f1b874b8ab0415d","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b6115104f1b874b8ab0415d: start","job_status":"start","queue_duration_s":0.78373,"scheduling_latency_s":0.783751} gitlab # [ 281.616860] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.711Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b6115104f1b874b8ab0415d","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b6115104f1b874b8ab0415d: done: 0.014858 sec","job_status":"done","queue_duration_s":0.78373,"scheduling_latency_s":0.783751,"redis_calls":13,"redis_duration_s":0.0028190000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00108,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001739,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009946,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014858,"completed_at":"2026-09-10T19:33:57.711Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.746520] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.713Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2ab05ad47320346f0806a4d7","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2ab05ad47320346f0806a4d7: start","job_status":"start","queue_duration_s":0.799834,"scheduling_latency_s":0.799855} gitlab # [ 281.748177] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.727Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2ab05ad47320346f0806a4d7","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2ab05ad47320346f0806a4d7: done: 0.014551 sec","job_status":"done","queue_duration_s":0.799834,"scheduling_latency_s":0.799855,"redis_calls":13,"redis_duration_s":0.002597,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000821,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001776,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009964,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014551,"completed_at":"2026-09-10T19:33:57.727Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.820445] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.729Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c16678a3cc8b50ded2d52ff","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c16678a3cc8b50ded2d52ff: start","job_status":"start","queue_duration_s":0.815633,"scheduling_latency_s":0.815654} gitlab # [ 281.821882] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.743Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c16678a3cc8b50ded2d52ff","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c16678a3cc8b50ded2d52ff: done: 0.01455 sec","job_status":"done","queue_duration_s":0.815633,"scheduling_latency_s":0.815654,"redis_calls":13,"redis_duration_s":0.0025830000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000831,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001752,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009937,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01455,"completed_at":"2026-09-10T19:33:57.743Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.905380] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.744Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7d7b86cecae0af8cf5b4753","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7d7b86cecae0af8cf5b4753: start","job_status":"start","queue_duration_s":0.831492,"scheduling_latency_s":0.831514} gitlab # [ 281.906459] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.759Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7d7b86cecae0af8cf5b4753","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7d7b86cecae0af8cf5b4753: done: 0.014659 sec","job_status":"done","queue_duration_s":0.831492,"scheduling_latency_s":0.831514,"redis_calls":13,"redis_duration_s":0.002672,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000845,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001827,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009953,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014659,"completed_at":"2026-09-10T19:33:57.759Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.976902] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.760Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70115a908f74dff4ad05bde5","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70115a908f74dff4ad05bde5: start","job_status":"start","queue_duration_s":0.847439,"scheduling_latency_s":0.847459} gitlab # [ 281.978168] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.775Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70115a908f74dff4ad05bde5","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70115a908f74dff4ad05bde5: done: 0.014556 sec","job_status":"done","queue_duration_s":0.847439,"scheduling_latency_s":0.847459,"redis_calls":13,"redis_duration_s":0.002642,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000826,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001816,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009899,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014556,"completed_at":"2026-09-10T19:33:57.775Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.060139] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.776Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c55f76585660814aa32d255","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c55f76585660814aa32d255: start","job_status":"start","queue_duration_s":0.863239,"scheduling_latency_s":0.863301} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses gitlab # [ 282.061478] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.791Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c55f76585660814aa32d255","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c55f76585660814aa32d255: done: 0.014661 sec","job_status":"done","queue_duration_s":0.863239,"scheduling_latency_s":0.863301,"redis_calls":13,"redis_duration_s":0.0025599999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000811,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001749,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009968,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014661,"completed_at":"2026-09-10T19:33:57.791Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.083429] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.792Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8424675f5382cd8557665476","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8424675f5382cd8557665476: start","job_status":"start","queue_duration_s":0.87916,"scheduling_latency_s":0.87918} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses, in 0.08 seconds) gitlab # [ 282.103267] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8424675f5382cd8557665476","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8424675f5382cd8557665476: done: 0.014609 sec","job_status":"done","queue_duration_s":0.87916,"scheduling_latency_s":0.87918,"redis_calls":13,"redis_duration_s":0.002567,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000777,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00179,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009943,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014609,"completed_at":"2026-09-10T19:33:57.807Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.172813] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.808Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"413efd996f3e6b8552f7b945","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-413efd996f3e6b8552f7b945: start","job_status":"start","queue_duration_s":0.895171,"scheduling_latency_s":0.895192} gitlab # [ 282.182201] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.823Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"413efd996f3e6b8552f7b945","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-413efd996f3e6b8552f7b945: done: 0.014619 sec","job_status":"done","queue_duration_s":0.895171,"scheduling_latency_s":0.895192,"redis_calls":13,"redis_duration_s":0.002595,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000784,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001811,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009915,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014619,"completed_at":"2026-09-10T19:33:57.823Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.243298] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.824Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d6fce914fb5cd1336eba20cd","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d6fce914fb5cd1336eba20cd: start","job_status":"start","queue_duration_s":0.911073,"scheduling_latency_s":0.911095} gitlab # [ 282.262424] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.839Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d6fce914fb5cd1336eba20cd","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d6fce914fb5cd1336eba20cd: done: 0.01455 sec","job_status":"done","queue_duration_s":0.911073,"scheduling_latency_s":0.911095,"redis_calls":13,"redis_duration_s":0.0025919999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00081,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001782,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009909,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01455,"completed_at":"2026-09-10T19:33:57.838Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.323922] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.840Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07ec733f6e06d21d1176dd62","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07ec733f6e06d21d1176dd62: start","job_status":"start","queue_duration_s":0.926864,"scheduling_latency_s":0.926886} gitlab # [ 282.343583] gitaly[931]: time="2026-09-10T19:34:01.944Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26D04XMR5HZ6J7F06CSAR9F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:34:31.643" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:34:01.943" grpc.time_ms=0.577 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 282.363276] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:34:01 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 51 gitlab # [ 282.366989] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.854Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07ec733f6e06d21d1176dd62","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07ec733f6e06d21d1176dd62: done: 0.014657 sec","job_status":"done","queue_duration_s":0.926864,"scheduling_latency_s":0.926886,"redis_calls":13,"redis_duration_s":0.002721,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000838,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001883,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014657,"completed_at":"2026-09-10T19:33:57.854Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.428092] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.856Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"161c51b18097a97f240c843f","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-161c51b18097a97f240c843f: start","job_status":"start","queue_duration_s":0.942773,"scheduling_latency_s":0.942793} gitlab # [ 282.447445] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.870Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"161c51b18097a97f240c843f","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-161c51b18097a97f240c843f: done: 0.014476 sec","job_status":"done","queue_duration_s":0.942773,"scheduling_latency_s":0.942793,"redis_calls":13,"redis_duration_s":0.0026,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000861,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001739,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009858,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014476,"completed_at":"2026-09-10T19:33:57.870Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.508775] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.871Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"edd2714387bbf6a755cce830","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-edd2714387bbf6a755cce830: start","job_status":"start","queue_duration_s":0.958547,"scheduling_latency_s":0.958568} gitlab # [ 282.528146] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.886Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"edd2714387bbf6a755cce830","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-edd2714387bbf6a755cce830: done: 0.014521 sec","job_status":"done","queue_duration_s":0.958547,"scheduling_latency_s":0.958568,"redis_calls":13,"redis_duration_s":0.002624,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000816,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001808,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009871,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014521,"completed_at":"2026-09-10T19:33:57.886Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.589050] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.887Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b9a6265f46543c79c203212","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b9a6265f46543c79c203212: start","job_status":"start","queue_duration_s":0.974349,"scheduling_latency_s":0.97437} gitlab # [ 282.608253] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.902Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b9a6265f46543c79c203212","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b9a6265f46543c79c203212: done: 0.014502 sec","job_status":"done","queue_duration_s":0.974349,"scheduling_latency_s":0.97437,"redis_calls":13,"redis_duration_s":0.002536,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00078,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001756,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009872,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014502,"completed_at":"2026-09-10T19:33:57.902Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.669210] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.903Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1543457fd6dbe529d59f4ce","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1543457fd6dbe529d59f4ce: start","job_status":"start","queue_duration_s":0.990107,"scheduling_latency_s":0.990128} gitlab # [ 282.688787] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.918Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1543457fd6dbe529d59f4ce","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1543457fd6dbe529d59f4ce: done: 0.014997 sec","job_status":"done","queue_duration_s":0.990107,"scheduling_latency_s":0.990128,"redis_calls":13,"redis_duration_s":0.0029159999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000888,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002028,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010096,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014997,"completed_at":"2026-09-10T19:33:57.918Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.750078] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.919Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d6045fd17deafe9a10390e6f","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d6045fd17deafe9a10390e6f: start","job_status":"start","queue_duration_s":1.006398,"scheduling_latency_s":1.006422} gitlab # [ 282.825347] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.934Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d6045fd17deafe9a10390e6f","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d6045fd17deafe9a10390e6f: done: 0.014789 sec","job_status":"done","queue_duration_s":1.006398,"scheduling_latency_s":1.006422,"redis_calls":13,"redis_duration_s":0.002653,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000826,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001827,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010033,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014789,"completed_at":"2026-09-10T19:33:57.934Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.829756] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.935Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7bec25deb80ce613ec764ecb","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7bec25deb80ce613ec764ecb: start","job_status":"start","queue_duration_s":1.022466,"scheduling_latency_s":1.022487} gitlab # [ 282.849162] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.950Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7bec25deb80ce613ec764ecb","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7bec25deb80ce613ec764ecb: done: 0.014893 sec","job_status":"done","queue_duration_s":1.022466,"scheduling_latency_s":1.022487,"redis_calls":13,"redis_duration_s":0.0027229999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000897,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001826,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014893,"completed_at":"2026-09-10T19:33:57.950Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.910319] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.952Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8dcbec1d7e5830973c495e3","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8dcbec1d7e5830973c495e3: start","job_status":"start","queue_duration_s":1.03864,"scheduling_latency_s":1.038661} gitlab # [ 282.929903] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.966Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8dcbec1d7e5830973c495e3","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8dcbec1d7e5830973c495e3: done: 0.014902 sec","job_status":"done","queue_duration_s":1.03864,"scheduling_latency_s":1.038661,"redis_calls":13,"redis_duration_s":0.002731,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000868,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001863,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009997,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014902,"completed_at":"2026-09-10T19:33:57.966Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.992559] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.968Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb9cf5761cb5ec9760949404","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb9cf5761cb5ec9760949404: start","job_status":"start","queue_duration_s":1.054795,"scheduling_latency_s":1.054816} gitlab # [ 283.066093] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.983Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb9cf5761cb5ec9760949404","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb9cf5761cb5ec9760949404: done: 0.01488 sec","job_status":"done","queue_duration_s":1.054795,"scheduling_latency_s":1.054816,"redis_calls":13,"redis_duration_s":0.0027010000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000812,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001889,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010107,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01488,"completed_at":"2026-09-10T19:33:57.983Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.073359] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.984Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c59851da01a454755a62bb31","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c59851da01a454755a62bb31: start","job_status":"start","queue_duration_s":1.070908,"scheduling_latency_s":1.07093} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses gitlab # [ 283.093407] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:57.999Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c59851da01a454755a62bb31","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c59851da01a454755a62bb31: done: 0.015326 sec","job_status":"done","queue_duration_s":1.070908,"scheduling_latency_s":1.07093,"redis_calls":13,"redis_duration_s":0.002897,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000891,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002006,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010065,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015326,"completed_at":"2026-09-10T19:33:57.999Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.178311] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:58.001Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e236c9f20ac1ceacddd9abe","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e236c9f20ac1ceacddd9abe: start","job_status":"start","queue_duration_s":1.087603,"scheduling_latency_s":1.087625} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses, in 0.09 seconds) gitlab # [ 283.179459] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:58.016Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e236c9f20ac1ceacddd9abe","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e236c9f20ac1ceacddd9abe: done: 0.015355 sec","job_status":"done","queue_duration_s":1.087603,"scheduling_latency_s":1.087625,"redis_calls":13,"redis_duration_s":0.0029649999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000859,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002106,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010135,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015355,"completed_at":"2026-09-10T19:33:58.016Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.267054] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:58.017Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f5935a627dfc845991f5957","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f5935a627dfc845991f5957: start","job_status":"start","queue_duration_s":1.104309,"scheduling_latency_s":1.104329} gitlab # [ 283.268169] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:58.032Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f5935a627dfc845991f5957","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f5935a627dfc845991f5957: done: 0.014843 sec","job_status":"done","queue_duration_s":1.104309,"scheduling_latency_s":1.104329,"redis_calls":13,"redis_duration_s":0.002676,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000824,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001852,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014843,"completed_at":"2026-09-10T19:33:58.032Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.349052] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:58.033Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b32361366fe5c3d5fd5c412c","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b32361366fe5c3d5fd5c412c: start","job_status":"start","queue_duration_s":1.120472,"scheduling_latency_s":1.120493} gitlab # [ 283.350394] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:33:58.048Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b32361366fe5c3d5fd5c412c","created_at":"2026-09-10T19:33:56.838Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-a4c1bff8a811415e","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"429922ff20f6f60290ef5321dbf1f3c9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-10T19:33:56.913Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b32361366fe5c3d5fd5c412c: done: 0.014898 sec","job_status":"done","queue_duration_s":1.120472,"scheduling_latency_s":1.120493,"redis_calls":13,"redis_duration_s":0.002732,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000958,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001774,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014898,"completed_at":"2026-09-10T19:33:58.048Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.419110] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:02.935Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"ce1c48bd0739f813eeda3e00","created_at":"2026-09-10T19:34:02.924Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9b8b00122b7860f0f0d8b0a9bfc79c0d","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:02.934Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-ce1c48bd0739f813eeda3e00: start","job_status":"start","queue_duration_s":0.001376,"scheduling_latency_s":0.001405} gitlab # [ 283.420737] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:34:03 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 62 gitlab # [ 283.423697] gitaly[931]: time="2026-09-10T19:34:03.028Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26D05ZA7B02DK0CMKM3NV0H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:34:33.026" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:34:03.027" grpc.time_ms=0.709 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 283.443862] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:03.017Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"ce1c48bd0739f813eeda3e00","created_at":"2026-09-10T19:34:02.924Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9b8b00122b7860f0f0d8b0a9bfc79c0d","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:02.934Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-ce1c48bd0739f813eeda3e00: done: 0.081825 sec","job_status":"done","queue_duration_s":0.001376,"scheduling_latency_s":0.001405,"redis_calls":11,"redis_duration_s":0.030416000000000002,"redis_read_bytes":9,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001441,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.023595,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00538,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014737,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001879568,"exclusive_lock_hold_duration_s":0.008632941,"duration_s":0.081825,"completed_at":"2026-09-10T19:34:03.017Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008877,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.556171] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:03.021Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"9e6e839388f241c1bb4ded5b","created_at":"2026-09-10T19:34:02.954Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f07a554479e4e84e68ca18e173fe1c5d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:02.958Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-9e6e839388f241c1bb4ded5b: start","job_status":"start","queue_duration_s":0.062307,"scheduling_latency_s":0.062331} gitlab # [ 283.647589] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:03.403Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"9e6e839388f241c1bb4ded5b","created_at":"2026-09-10T19:34:02.954Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f07a554479e4e84e68ca18e173fe1c5d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:02.958Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-9e6e839388f241c1bb4ded5b: done: 0.382477 sec","job_status":"done","queue_duration_s":0.062307,"scheduling_latency_s":0.062331,"redis_calls":11,"redis_duration_s":0.014946000000000001,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.007859,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003635,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003452,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.071,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":130,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.071,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.130001,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000787531,"exclusive_lock_hold_duration_s":0.346694901,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.382477,"completed_at":"2026-09-10T19:34:03.403Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.2461,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.717275] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:03.473Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"923cfdb0d852422f4ae3c972","created_at":"2026-09-10T19:34:02.989Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"325b7ad996271953b734d9c4e0d4014c","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:02.996Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-923cfdb0d852422f4ae3c972: start","job_status":"start","queue_duration_s":0.477217,"scheduling_latency_s":0.477255} gitlab # [ 283.720586] gitaly[931]: time="2026-09-10T19:34:03.477Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26D06BBBAP3M3SAPY8WE8S3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:34:33.475" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:34:03.476" grpc.time_ms=0.551 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 283.793717] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:03.549Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"923cfdb0d852422f4ae3c972","created_at":"2026-09-10T19:34:02.989Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"325b7ad996271953b734d9c4e0d4014c","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:02.996Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-923cfdb0d852422f4ae3c972: done: 0.076228 sec","job_status":"done","queue_duration_s":0.477217,"scheduling_latency_s":0.477255,"redis_calls":7,"redis_duration_s":0.0032389999999999997,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002087,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001152,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022769,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.076228,"completed_at":"2026-09-10T19:34:03.549Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004186,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.797775] gitaly[931]: time="2026-09-10T19:34:03.554Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M26D06BBBAP3M3SAPY8WE8S3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-10T19:34:13.132" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:34:03.542" grpc.time_ms=11.198 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 283.880202] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:03.636Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"e018f095a39795142ed27100","created_at":"2026-09-10T19:34:03.001Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"261d710fbee4acac0a610f3290e27466","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:03.008Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-e018f095a39795142ed27100: start","job_status":"start","queue_duration_s":0.627596,"scheduling_latency_s":0.627654} gitlab # [ 283.942104] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:03.697Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"e018f095a39795142ed27100","created_at":"2026-09-10T19:34:03.001Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"261d710fbee4acac0a610f3290e27466","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:03.008Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-e018f095a39795142ed27100: done: 0.061396 sec","job_status":"done","queue_duration_s":0.627596,"scheduling_latency_s":0.627654,"redis_calls":9,"redis_duration_s":0.003118,"redis_read_bytes":7,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000745,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001627,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000746,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021349,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.061396,"completed_at":"2026-09-10T19:34:03.697Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003814,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.973211] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:34:03 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/3cbe2f74443ff0f3ea476d1bcddd263441bc5445 HTTP/1.1" 201 676 "" "python-httpx/0.28.1" 357 gitlab # [ 283.976228] nixbot[2723]: {"ts": "2026-09-10T19:34:03.731Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/3cbe2f74443ff0f3ea476d1bcddd263441bc5445 \"HTTP/1.1 201 Created\""} gitlab # [ 284.017727] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:03.773Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"103e124f80140e3de50e8995","created_at":"2026-09-10T19:34:03.018Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"34f8e579aea0adf157c1e957b1e5dd9b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:03.019Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-103e124f80140e3de50e8995: start","job_status":"start","queue_duration_s":0.753147,"scheduling_latency_s":0.753185} gitlab # [ 284.034096] gitaly[931]: time="2026-09-10T19:34:03.790Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26D06PT584Q3KH4H9QS5Z2A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:34:33.588" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:34:03.789" grpc.time_ms=0.897 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 284.073860] gitaly[931]: time="2026-09-10T19:34:03.830Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26D06PT584Q3KH4H9QS5Z2A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-10T19:34:13.273" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:34:03.823" grpc.time_ms=6.696 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 284.133243] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:34:03 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/3cbe2f74443ff0f3ea476d1bcddd263441bc5445 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 150 gitlab # [ 284.137981] nixbot[2723]: {"ts": "2026-09-10T19:34:03.890Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/3cbe2f74443ff0f3ea476d1bcddd263441bc5445 \"HTTP/1.1 201 Created\""} gitlab # [ 284.140170] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:03.894Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"103e124f80140e3de50e8995","created_at":"2026-09-10T19:34:03.018Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"34f8e579aea0adf157c1e957b1e5dd9b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:03.019Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-103e124f80140e3de50e8995: done: 0.121924 sec","job_status":"done","queue_duration_s":0.753147,"scheduling_latency_s":0.753185,"redis_calls":9,"redis_duration_s":0.025769,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.023381,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002388,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036064,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00129821,"exclusive_lock_hold_duration_s":0.025902454,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.121924,"completed_at":"2026-09-10T19:34:03.894Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012169,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.142684] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:03.898Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"d2f6a4eabd0474b9e39eceaf","created_at":"2026-09-10T19:34:03.027Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d2771fed89d63e8c1b6035386152f679","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-10T19:34:03.032Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-d2f6a4eabd0474b9e39eceaf: start","job_status":"start","queue_duration_s":0.866543,"scheduling_latency_s":0.866606} gitlab # [ 284.163344] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:03.912Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses gitlab # [ 284.167957] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:03.917Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"d2f6a4eabd0474b9e39eceaf","created_at":"2026-09-10T19:34:03.027Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d2771fed89d63e8c1b6035386152f679","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-10T19:34:03.032Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-d2f6a4eabd0474b9e39eceaf: done: 0.018689 sec","job_status":"done","queue_duration_s":0.866543,"scheduling_latency_s":0.866606,"redis_calls":7,"redis_duration_s":0.0018679999999999999,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001331,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000537,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011433,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018689,"completed_at":"2026-09-10T19:34:03.917Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.229458] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:03.918Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"70ca3f906343f364818bafa9","created_at":"2026-09-10T19:34:03.046Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5c82f2befca2bf793bf4d0824cbf0c16","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-10T19:34:03.049Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-70ca3f906343f364818bafa9: start","job_status":"start","queue_duration_s":0.868939,"scheduling_latency_s":0.86896} gitlab # [ 284.250555] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:03.936Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"70ca3f906343f364818bafa9","created_at":"2026-09-10T19:34:03.046Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5c82f2befca2bf793bf4d0824cbf0c16","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-10T19:34:03.049Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-70ca3f906343f364818bafa9: done: 0.017572 sec","job_status":"done","queue_duration_s":0.868939,"scheduling_latency_s":0.86896,"redis_calls":7,"redis_duration_s":0.0018870000000000002,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001381,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000506,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012184,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017572,"completed_at":"2026-09-10T19:34:03.936Z","load_balancing_strategy":"primary","db_duration_s":0.000837,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.355262] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:03.937Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"97f79791168d6116a07558f9","created_at":"2026-09-10T19:34:03.056Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4dc9cd2ef8a41f42afa5d74f09a93da1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:03.058Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-97f79791168d6116a07558f9: start","job_status":"start","queue_duration_s":0.879238,"scheduling_latency_s":0.87926} gitlab # [ 284.356796] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:03.945Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses, in 0.12 seconds) gitlab # [ 284.357935] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:03.950Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"97f79791168d6116a07558f9","created_at":"2026-09-10T19:34:03.056Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4dc9cd2ef8a41f42afa5d74f09a93da1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:03.058Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-97f79791168d6116a07558f9: done: 0.012843 sec","job_status":"done","queue_duration_s":0.879238,"scheduling_latency_s":0.87926,"redis_calls":7,"redis_duration_s":0.002749,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002105,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000644,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007722,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012843,"completed_at":"2026-09-10T19:34:03.950Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.425974] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:03.952Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"64a1f388ea1f350cd89ba959","created_at":"2026-09-10T19:34:03.067Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6472661cf673bc105818de7bf10eae30","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:03.070Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-64a1f388ea1f350cd89ba959: start","job_status":"start","queue_duration_s":0.881871,"scheduling_latency_s":0.881894} gitlab # [ 284.427963] gitaly[931]: time="2026-09-10T19:34:04.146Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26D071E97M47SNZ9ZWK4EC0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:34:34.144" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:34:04.145" grpc.time_ms=0.855 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 284.503380] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:34:04 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses HTTP/1.1" 200 1314 "" "curl/8.21.0" 86 gitlab # [ 284.620546] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:04.375Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"64a1f388ea1f350cd89ba959","created_at":"2026-09-10T19:34:03.067Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6472661cf673bc105818de7bf10eae30","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:03.070Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-64a1f388ea1f350cd89ba959: done: 0.422326 sec","job_status":"done","queue_duration_s":0.881871,"scheduling_latency_s":0.881894,"redis_calls":562,"redis_duration_s":0.139063,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001613,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.13745,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.32252,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.422326,"completed_at":"2026-09-10T19:34:04.374Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.686268] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:04.378Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"7f9f7636866d076cbc2e4b9f","created_at":"2026-09-10T19:34:03.078Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c17c7f8352f1cca445bdf9522e58d12e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:03.081Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-7f9f7636866d076cbc2e4b9f: start","job_status":"start","queue_duration_s":1.297527,"scheduling_latency_s":1.2976} gitlab # [ 284.687459] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:04.388Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 284.688299] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:04.393Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"7f9f7636866d076cbc2e4b9f","created_at":"2026-09-10T19:34:03.078Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c17c7f8352f1cca445bdf9522e58d12e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:03.081Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-7f9f7636866d076cbc2e4b9f: done: 0.015087 sec","job_status":"done","queue_duration_s":1.297527,"scheduling_latency_s":1.2976,"redis_calls":7,"redis_duration_s":0.002521,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001722,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000799,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008681,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015087,"completed_at":"2026-09-10T19:34:04.393Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.735865] gitaly[931]: time="2026-09-10T19:34:04.408Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26D077S7DBE0A0E2RKS1ACE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-10T19:34:34.206" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-10T19:34:04.407" grpc.time_ms=0.773 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 284.809288] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:04.395Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"37ed36de6a175dd81c02a076","created_at":"2026-09-10T19:34:03.090Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e05bee9d9d54635a246a1049ec61795c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:03.092Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-37ed36de6a175dd81c02a076: start","job_status":"start","queue_duration_s":1.302924,"scheduling_latency_s":1.302949} gitlab # [ 284.810449] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:04.404Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 284.811287] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:04.411Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"37ed36de6a175dd81c02a076","created_at":"2026-09-10T19:34:03.090Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e05bee9d9d54635a246a1049ec61795c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:03.092Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-37ed36de6a175dd81c02a076: done: 0.015998 sec","job_status":"done","queue_duration_s":1.302924,"scheduling_latency_s":1.302949,"redis_calls":7,"redis_duration_s":0.002699,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00203,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000669,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009144,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015998,"completed_at":"2026-09-10T19:34:04.411Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.891643] gitaly[931]: time="2026-09-10T19:34:04.440Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M26D077S7DBE0A0E2RKS1ACE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-10T19:34:14.073" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-10T19:34:04.434" grpc.time_ms=6.553 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 284.893134] nixbot[2723]: {"ts": "2026-09-10T19:34:04.600Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/3cbe2f74443ff0f3ea476d1bcddd263441bc5445 \"HTTP/1.1 201 Created\""} gitlab # [ 284.893930] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:34:04 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/3cbe2f74443ff0f3ea476d1bcddd263441bc5445 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 317 gitlab # [ 284.895736] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:04.414Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"d83dabbd56067a34efc45929","created_at":"2026-09-10T19:34:03.184Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d63ac8a2d194bba9801e81bb802f6c3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:03.188Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-d83dabbd56067a34efc45929: start","job_status":"start","queue_duration_s":1.226191,"scheduling_latency_s":1.226231} gitlab # [ 284.898213] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:04.431Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"d83dabbd56067a34efc45929","created_at":"2026-09-10T19:34:03.184Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d63ac8a2d194bba9801e81bb802f6c3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:03.188Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-d83dabbd56067a34efc45929: done: 0.01725 sec","job_status":"done","queue_duration_s":1.226191,"scheduling_latency_s":1.226231,"redis_calls":7,"redis_duration_s":0.0017000000000000001,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001164,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000536,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010226,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01725,"completed_at":"2026-09-10T19:34:04.431Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.972329] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:04.432Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"e64a69ff20573827e19f3de5","created_at":"2026-09-10T19:34:03.196Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1ad1a74312d9dbb7fd6564249796a265","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:03.200Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-e64a69ff20573827e19f3de5: start","job_status":"start","queue_duration_s":1.232224,"scheduling_latency_s":1.232246} gitlab # [ 284.986471] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:04.451Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"e64a69ff20573827e19f3de5","created_at":"2026-09-10T19:34:03.196Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1ad1a74312d9dbb7fd6564249796a265","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:03.200Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-e64a69ff20573827e19f3de5: done: 0.018599 sec","job_status":"done","queue_duration_s":1.232224,"scheduling_latency_s":1.232246,"redis_calls":10,"redis_duration_s":0.002609,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001626,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000983,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011807,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.003573638,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.018599,"completed_at":"2026-09-10T19:34:04.451Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001304,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 285.067718] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:04.452Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"c6348ef4a18fa6d4c778215c","created_at":"2026-09-10T19:34:03.211Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"baf235991acca91b3731f816eda6ef81","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:03.213Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-c6348ef4a18fa6d4c778215c: start","job_status":"start","queue_duration_s":1.239629,"scheduling_latency_s":1.239653} gitlab # [ 285.071355] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:04.466Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"c6348ef4a18fa6d4c778215c","created_at":"2026-09-10T19:34:03.211Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"baf235991acca91b3731f816eda6ef81","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:03.213Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-c6348ef4a18fa6d4c778215c: done: 0.013832 sec","job_status":"done","queue_duration_s":1.239629,"scheduling_latency_s":1.239653,"redis_calls":7,"redis_duration_s":0.001673,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001171,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000502,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008357,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013832,"completed_at":"2026-09-10T19:34:04.466Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00036,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.178884] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:04.468Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"4483b3c55d0db3fb9fe1c072","created_at":"2026-09-10T19:34:03.221Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee5003315496778be375d70ac3588827","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:03.224Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-4483b3c55d0db3fb9fe1c072: start","job_status":"start","queue_duration_s":1.244034,"scheduling_latency_s":1.244056} gitlab # [ 285.179961] gitaly[931]: time="2026-09-10T19:34:04.812Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M26D06BBBAP3M3SAPY8WE8S3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-10T19:34:04.812" grpc.time_ms=0.372 pid=931 span.kind=server system=grpc gitlab # [ 285.181194] gitaly[931]: time="2026-09-10T19:34:04.868Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M26D07R8NW5C8BYMGGTNQ8EM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:34:04.861" grpc.time_ms=6.027 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 285.189777] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:34:04 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 61 gitlab # [ 285.193075] gitlab-workhorse[830]: time="2026-09-10T19:34:04Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 285.196133] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:34:04 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 45 gitlab # [ 285.199207] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:04.489Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"4483b3c55d0db3fb9fe1c072","created_at":"2026-09-10T19:34:03.221Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee5003315496778be375d70ac3588827","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:03.224Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-4483b3c55d0db3fb9fe1c072: done: 0.021181 sec","job_status":"done","queue_duration_s":1.244034,"scheduling_latency_s":1.244056,"redis_calls":14,"redis_duration_s":0.0032500000000000003,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000323,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002216,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000711,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014653,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021181,"completed_at":"2026-09-10T19:34:04.489Z","load_balancing_strategy":"primary","db_duration_s":0.000445,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.275180] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:04.491Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"3addb7d16974198a61586025","created_at":"2026-09-10T19:34:03.229Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ad1d6179863794e74560a412054ebf9b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:03.231Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-3addb7d16974198a61586025: start","job_status":"start","queue_duration_s":1.259367,"scheduling_latency_s":1.259389} gitlab # [ 285.285499] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:04.498Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 285.290870] gitaly[931]: time="2026-09-10T19:34:04.917Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M26D07T7ZCYZNYDZW8AM8C4K grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:34:04.909" pid=931 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses gitlab # [ 285.361935] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:04.503Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"3addb7d16974198a61586025","created_at":"2026-09-10T19:34:03.229Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ad1d6179863794e74560a412054ebf9b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:03.231Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-3addb7d16974198a61586025: done: 0.012079 sec","job_status":"done","queue_duration_s":1.259367,"scheduling_latency_s":1.259389,"redis_calls":7,"redis_duration_s":0.001719,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001174,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000545,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007375,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012079,"completed_at":"2026-09-10T19:34:04.503Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.432795] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:04.504Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"90e0fc7ebc3d2251354845d7","created_at":"2026-09-10T19:34:03.240Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3120dc6563466a617ed1e775bf095159","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:03.248Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-90e0fc7ebc3d2251354845d7: start","job_status":"start","queue_duration_s":1.256034,"scheduling_latency_s":1.256055} gitlab # [ 285.433879] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:34:04 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 47 gitlab # [ 285.434677] gitlab-workhorse[830]: time="2026-09-10T19:34:05Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 285.435808] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:34:05 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 52 gitlab # [ 285.437101] gitaly[931]: time="2026-09-10T19:34:04.917Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26D07T7ZCYZNYDZW8AM8C4K grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:34:04.909" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses, in 0.13 seconds) (finished: subtest: eval and build post commit statuses, in 13.02 seconds) (finished: run the VM test script, in 371.90 seconds) gitlab # [ 285.437894] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:04.521Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"90e0fc7ebc3d2251354845d7","created_at":"2026-09-10T19:34:03.240Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3120dc6563466a617ed1e775bf095159","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:03.248Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-90e0fc7ebc3d2251354845d7: done: 0.017347 sec","job_status":"done","queue_duration_s":1.256034,"scheduling_latency_s":1.256055,"redis_calls":11,"redis_duration_s":0.0026810000000000002,"redis_read_bytes":8,"redis_write_bytes":1094,"redis_cache_calls":2,"redis_cache_duration_s":0.000595,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000436,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001202,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":482,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000448,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010134,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017347,"completed_at":"2026-09-10T19:34:04.521Z","load_balancing_strategy":"primary","db_duration_s":0.000401,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.539401] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:04.523Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"3a7f7f9c1bbbb23717fcfd9c","created_at":"2026-09-10T19:34:03.266Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8952b605cfd6c9f46809cfe9b3d2eb04","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:03.268Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-3a7f7f9c1bbbb23717fcfd9c: start","job_status":"start","queue_duration_s":1.254557,"scheduling_latency_s":1.254579} gitlab # [ 285.540437] gitaly[931]: time="2026-09-10T19:34:04.917Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M26D07T7ZCYZNYDZW8AM8C4K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-10T19:34:04.909" grpc.time_ms=7.501 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=931 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 285.541206] gitlab-workhorse[830]: gitlab 2001:db8:1::1 - - [2026/09/10:19:34:05 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3cbe2f74443ff0f3ea476d1bcddd263441bc5445/statuses HTTP/1.1" 200 1335 "" "curl/8.21.0" 98 test script finished in 372.00s cleanup kill QemuMachine (pid 46) gitlab # [ 285.542002] sidekiq-cluster[2598]: {"severity":"INFO","time":"2026-09-10T19:34:04.547Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"3a7f7f9c1bbbb23717fcfd9c","created_at":"2026-09-10T19:34:03.266Z","trace_propagation_headers":{"sentry-trace":"d184d1c746b44a33a541d41f50c80616-5915ae78d87647ca","baggage":"sentry-trace_id=d184d1c746b44a33a541d41f50c80616,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8952b605cfd6c9f46809cfe9b3d2eb04","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-10T19:34:03.268Z","job_size_bytes":2,"pid":2598,"sidekiq_tid":"m836","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-3a7f7f9c1bbbb23717fcfd9c: done: 0.024193 sec","job_status":"done","queue_duration_s":1.254557,"scheduling_latency_s":1.254579,"redis_calls":9,"redis_duration_s":0.002307,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001193,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001114,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/gxzhl7aaiid7zp3y47jqqiq7zg5mqpwp-python3-3.14.6/bin/python3.14) (finished: cleanup, in 1.24 seconds) additionally exposed symbols: gitlab, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh [] [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-09-10T19:32:56.862Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] [] [] [] [] [] [] [] [] [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '3cbe2f74443ff0f3ea476d1bcddd263441bc5445', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-09-10T19:33:52.157036Z', 'started_at': None, 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': {'position': 1, 'blocked_by': []}}] [] [{'id': 5, 'sha': '3cbe2f74443ff0f3ea476d1bcddd263441bc5445', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-10T19:33:52.545Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '3cbe2f74443ff0f3ea476d1bcddd263441bc5445', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-10T19:33:52.545Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '3cbe2f74443ff0f3ea476d1bcddd263441bc5445', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-10T19:33:52.545Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '3cbe2f74443ff0f3ea476d1bcddd263441bc5445', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-10T19:33:52.545Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '3cbe2f74443ff0f3ea476d1bcddd263441bc5445', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-10T19:33:52.545Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '3cbe2f74443ff0f3ea476d1bcddd263441bc5445', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-10T19:33:52.545Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '3cbe2f74443ff0f3ea476d1bcddd263441bc5445', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-10T19:33:52.545Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '3cbe2f74443ff0f3ea476d1bcddd263441bc5445', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-10T19:33:52.545Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '3cbe2f74443ff0f3ea476d1bcddd263441bc5445', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-10T19:33:52.545Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '3cbe2f74443ff0f3ea476d1bcddd263441bc5445', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 10s (1 warning)', 'created_at': '2026-09-10T19:33:52.545Z', 'started_at': None, 'finished_at': '2026-09-10T19:34:03.690Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '3cbe2f74443ff0f3ea476d1bcddd263441bc5445', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-09-10T19:34:03.869Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '3cbe2f74443ff0f3ea476d1bcddd263441bc5445', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 10s (1 warning)', 'created_at': '2026-09-10T19:33:52.545Z', 'started_at': None, 'finished_at': '2026-09-10T19:34:03.690Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '3cbe2f74443ff0f3ea476d1bcddd263441bc5445', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-09-10T19:34:03.869Z', 'started_at': None, 'finished_at': '2026-09-10T19:34:04.459Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]