tribuchet: building on eliza Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 3600 seconds run the VM test script start all VMs gitlab: starting vm gitlab # Disk image does not exist, creating the virtualisation disk image... gitlab: QEMU running (pid 46) gitlab # Formatting '/build/vm-state-gitlab/tmp.UCN4s7vFQA', fmt=raw size=1073741824 gitlab # mke2fs 1.47.4 (6-Mar-2025) gitlab # Discarding device blocks: 0/262144 done (finished: start all VMs, in 70.91 seconds) gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes gitlab: waiting for unit setup-gitlab.service gitlab # Filesystem UUID: 3c54de56-811b-4570-9652-a6366a149d56 gitlab: waiting for the VM to finish booting gitlab # Superblock backups stored on blocks: gitlab # 32768, 98304, 163840, 229376 gitlab # gitlab # Allocating group tables: 0/8 done gitlab # Writing inode tables: 0/8 done gitlab # Creating journal (8192 blocks): done gitlab # Writing superblocks and filesystem accounting information: 0/8 done gitlab # gitlab # Virtualisation disk image created. gitlab # Creating Nix store image... gitlab # Created Nix store image. gitlab # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] gitlab # [ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Thu Aug 6 17:27:09 UTC 2026 gitlab # [ 0.000000] KASLR enabled gitlab # [ 0.000000] random: crng init done gitlab # [ 0.000000] Machine model: linux,dummy-virt gitlab # [ 0.000000] efi: UEFI not found. gitlab # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT gitlab # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff] gitlab # [ 0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf] gitlab # [ 0.000000] Zone ranges: gitlab # [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff] gitlab # [ 0.000000] DMA32 empty gitlab # [ 0.000000] Normal [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.000000] Device empty gitlab # [ 0.000000] Movable zone start for each node gitlab # [ 0.000000] Early memory node ranges gitlab # [ 0.000000] node 0: [mem 0x0000000040000000-0x00000001bfffffff] gitlab # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff] gitlab # [ 0.000000] cma: Reserved 32 MiB at 0x00000000fe000000 gitlab # [ 0.000000] psci: probing for conduit method from DT. gitlab # [ 0.000000] psci: PSCIv1.3 detected in firmware. gitlab # [ 0.000000] psci: Using standard PSCI v0.2 function IDs gitlab # [ 0.000000] psci: Trusted OS migration not required gitlab # [ 0.000000] psci: SMC Calling Convention v1.1 gitlab # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) gitlab # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 gitlab # [ 0.000000] Detected PIPT I-cache on CPU0 gitlab # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) gitlab # [ 0.000000] CPU features: detected: GICv3 CPU interface gitlab # [ 0.000000] CPU features: detected: Spectre-v4 gitlab # [ 0.000000] CPU features: detected: Spectre-BHB gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 gitlab # [ 0.000000] alternatives: applying boot alternatives gitlab # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/1lnw6hp2c14n58kyqwr96rnmq7dapmi1-nixos-system-gitlab-test/init regInfo=/nix/store/88h4l4jvg1ia0nd8y94m6k40ygvalvld-closure-info/registration console=ttyAMA0,115200n8 console=tty0 gitlab # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/88h4l4jvg1ia0nd8y94m6k40ygvalvld-closure-info/registration", will be passed to user space. gitlab # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes gitlab # [ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) gitlab # [ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) gitlab # [ 0.000000] software IO TLB: area num 4. gitlab # [ 0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB) gitlab # [ 0.000000] Fallback order for Node 0: 0 gitlab # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1572864 gitlab # [ 0.000000] Policy zone: Normal gitlab # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off gitlab # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 gitlab # [ 0.000000] allocated 12582912 bytes of page_ext gitlab # [ 0.000000] ftrace: allocating 74867 entries in 294 pages gitlab # [ 0.000000] ftrace: allocated 294 pages with 4 groups gitlab # [ 0.000000] rcu: Hierarchical RCU implementation. gitlab # [ 0.000000] rcu: RCU event tracing is enabled. gitlab # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. gitlab # [ 0.000000] Trampoline variant of Tasks RCU enabled. gitlab # [ 0.000000] Rude variant of Tasks RCU enabled. gitlab # [ 0.000000] Tracing variant of Tasks RCU enabled. gitlab # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. gitlab # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 gitlab # [ 0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 gitlab # [ 0.000000] GICv3: 256 SPIs implemented gitlab # [ 0.000000] GICv3: 0 Extended SPIs implemented gitlab # [ 0.000000] Root IRQ handler: gic_handle_irq gitlab # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI gitlab # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 gitlab # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 gitlab # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f10000 (indirect, esz 8, psz 64K, shr 1) gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f20000 (flat, esz 8, psz 64K, shr 1) gitlab # [ 0.000000] GICv3: using LPI property table @0x0000000100f40000 gitlab # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f50000 gitlab # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. gitlab # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns gitlab # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). gitlab # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns gitlab # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns gitlab # [ 0.000033] arm-pv: using stolen time PV gitlab # [ 0.000420] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) gitlab # [ 0.000647] Console: colour dummy device 80x25 gitlab # [ 0.000654] printk: legacy console [tty0] enabled gitlab # [ 0.000859] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) gitlab # [ 0.000867] pid_max: default: 32768 minimum: 301 gitlab # [ 0.000947] LSM: initializing lsm=capability,landlock,yama,bpf,ima gitlab # [ 0.001115] landlock: Up and running. gitlab # [ 0.001119] Yama: becoming mindful. gitlab # [ 0.001571] LSM support for eBPF active gitlab # [ 0.001789] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.001905] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.003239] cacheinfo: Unable to detect cache hierarchy for CPU 0 gitlab # [ 0.004036] rcu: Hierarchical SRCU implementation. gitlab # [ 0.004042] rcu: Max phase no-delay instances is 1000. gitlab # [ 0.004221] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level gitlab # [ 0.005326] fsl-mc MSI: its@8080000 domain created gitlab # [ 0.005424] EFI services will not be available. gitlab # [ 0.005607] smp: Bringing up secondary CPUs ... gitlab # [ 0.006518] Detected PIPT I-cache on CPU1 gitlab # [ 0.006552] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000 gitlab # [ 0.006593] GICv3: CPU1: using allocated LPI pending table @0x0000000100f60000 gitlab # [ 0.006639] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40] gitlab # [ 0.007565] Detected PIPT I-cache on CPU2 gitlab # [ 0.007632] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000 gitlab # [ 0.007707] GICv3: CPU2: using allocated LPI pending table @0x0000000100f70000 gitlab # [ 0.007781] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40] gitlab # [ 0.009212] Detected PIPT I-cache on CPU3 gitlab # [ 0.009287] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000 gitlab # [ 0.009364] GICv3: CPU3: using allocated LPI pending table @0x0000000100f80000 gitlab # [ 0.009438] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40] gitlab # [ 0.010101] smp: Brought up 1 node, 4 CPUs gitlab # [ 0.010132] SMP: Total of 4 processors activated. gitlab # [ 0.010135] CPU: All CPU(s) started at EL1 gitlab # [ 0.010144] CPU features: detected: Branch Target Identification gitlab # [ 0.010148] CPU features: detected: ARMv8.4 Translation Table Level gitlab # [ 0.010151] CPU features: detected: Instruction cache invalidation not required for I/D coherence gitlab # [ 0.010154] CPU features: detected: Data cache clean to the PoU not required for I/D coherence gitlab # [ 0.010158] CPU features: detected: Common not Private translations gitlab # [ 0.010160] CPU features: detected: CRC32 instructions gitlab # [ 0.010162] CPU features: detected: Data cache clean to Point of Deep Persistence gitlab # [ 0.010166] CPU features: detected: Data cache clean to Point of Persistence gitlab # [ 0.010168] CPU features: detected: Data independent timing control (DIT) gitlab # [ 0.010171] CPU features: detected: E0PD gitlab # [ 0.010173] CPU features: detected: Enhanced Counter Virtualization gitlab # [ 0.010176] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) gitlab # [ 0.010179] CPU features: detected: Enhanced Virtualization Traps gitlab # [ 0.010182] CPU features: detected: Fine Grained Traps gitlab # [ 0.010185] CPU features: detected: Generic authentication (architected QARMA5 algorithm) gitlab # [ 0.010189] CPU features: detected: RCpc load-acquire (LDAPR) gitlab # [ 0.010192] CPU features: detected: LSE atomic instructions gitlab # [ 0.010194] CPU features: detected: Privileged Access Never gitlab # [ 0.010197] CPU features: detected: PMUv3 gitlab # [ 0.010199] CPU features: detected: RAS Extension Support gitlab # [ 0.010201] CPU features: detected: RASv1p1 Extension Support gitlab # [ 0.010204] CPU features: detected: Random Number Generator gitlab # [ 0.010206] CPU features: detected: Speculation barrier (SB) gitlab # [ 0.010208] CPU features: detected: Stage-2 Force Write-Back gitlab # [ 0.010211] CPU features: detected: TLB range maintenance instructions gitlab # [ 0.010214] CPU features: detected: Speculative Store Bypassing Safe (SSBS) gitlab # [ 0.010310] alternatives: applying system-wide alternatives gitlab # [ 0.013411] CPU features: detected: BBM Level 2 without TLB conflict abort gitlab # [ 0.013662] Memory: 5963752K/6291456K available (24448K kernel code, 7090K rwdata, 26340K rodata, 4736K init, 1106K bss, 272908K reserved, 32768K cma-reserved) gitlab # [ 0.015142] devtmpfs: initialized gitlab # [ 0.019180] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) gitlab # [ 0.019253] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). gitlab # [ 0.019490] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL gitlab # [ 0.019497] 0 pages in range for non-PLT usage gitlab # [ 0.019499] 508288 pages in range for PLT usage gitlab # [ 0.019668] pinctrl core: initialized pinctrl subsystem gitlab # [ 0.020632] DMI not present or invalid. gitlab # [ 0.024181] NET: Registered PF_NETLINK/PF_ROUTE protocol family gitlab # [ 0.026810] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations gitlab # [ 0.027081] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations gitlab # [ 0.029087] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations gitlab # [ 0.029119] audit: initializing netlink subsys (disabled) gitlab # [ 0.029330] audit: type=2000 audit(0.024:1): state=initialized audit_enabled=0 res=1 gitlab # [ 0.029859] thermal_sys: Registered thermal governor 'fair_share' gitlab # [ 0.029861] thermal_sys: Registered thermal governor 'bang_bang' gitlab # [ 0.029865] thermal_sys: Registered thermal governor 'step_wise' gitlab # [ 0.029868] thermal_sys: Registered thermal governor 'user_space' gitlab # [ 0.029870] thermal_sys: Registered thermal governor 'power_allocator' gitlab # [ 0.029927] cpuidle: using governor ladder gitlab # [ 0.029945] cpuidle: using governor menu gitlab # [ 0.030150] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. gitlab # [ 0.030286] ASID allocator initialised with 65536 entries gitlab # [ 0.031960] Serial: AMBA PL011 UART driver gitlab # [ 0.037551] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 gitlab # [ 0.037934] printk: console [ttyAMA0] enabled gitlab # [ 0.044296] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages gitlab # [ 0.044309] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page gitlab # [ 0.044315] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages gitlab # [ 0.044319] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page gitlab # [ 0.044323] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages gitlab # [ 0.044328] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page gitlab # [ 0.044332] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages gitlab # [ 0.044336] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page gitlab # [ 0.051959] fbcon: Taking over console gitlab # [ 0.051980] ACPI: Interpreter disabled. gitlab # [ 0.053566] iommu: Default domain type: Translated gitlab # [ 0.053573] iommu: DMA domain TLB invalidation policy: strict mode gitlab # [ 0.066088] SCSI subsystem initialized gitlab # [ 0.066684] usbcore: registered new interface driver usbfs gitlab # [ 0.066766] usbcore: registered new interface driver hub gitlab # [ 0.066794] usbcore: registered new device driver usb gitlab # [ 0.067241] pps_core: LinuxPPS API ver. 1 registered gitlab # [ 0.067248] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti gitlab # [ 0.067260] PTP clock support registered gitlab # [ 0.067322] EDAC MC: Ver: 3.0.0 gitlab # [ 0.067713] scmi_core: SCMI protocol bus registered gitlab # [ 0.068347] FPGA manager framework gitlab # [ 0.069619] vgaarb: loaded gitlab # [ 0.070475] clocksource: Switched to clocksource arch_sys_counter gitlab # [ 0.073023] VFS: Disk quotas dquot_6.6.0 gitlab # [ 0.073066] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) gitlab # [ 0.073641] netfs: FS-Cache loaded gitlab # [ 0.073817] pnp: PnP ACPI: disabled gitlab # [ 0.079683] NET: Registered PF_INET protocol family gitlab # [ 0.080900] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) gitlab # [ 0.110159] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) gitlab # [ 0.110214] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) gitlab # [ 0.110264] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) gitlab # [ 0.110622] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) gitlab # [ 0.111172] TCP: Hash tables configured (established 65536 bind 65536) gitlab # [ 0.111284] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) gitlab # [ 0.111360] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.111473] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.111664] NET: Registered PF_UNIX/PF_LOCAL protocol family gitlab # [ 0.111690] NET: Registered PF_XDP protocol family gitlab # [ 0.111707] PCI: CLS 0 bytes, default 64 gitlab # [ 0.111891] Trying to unpack rootfs image as initramfs... gitlab # [ 0.122599] kvm [1]: HYP mode not available gitlab # [ 0.166748] Initialise system trusted keyrings gitlab # [ 0.167321] workingset: timestamp_bits=42 max_order=21 bucket_order=0 gitlab # [ 0.167827] squashfs: version 4.0 (2009/01/31) Phillip Lougher gitlab # [ 0.167886] 9p: Installing v9fs 9p2000 file system support gitlab # [ 0.179854] Key type asymmetric registered gitlab # [ 0.179859] Asymmetric key parser 'x509' registered gitlab # [ 0.179910] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) gitlab # [ 0.180058] io scheduler mq-deadline registered gitlab # [ 0.180063] io scheduler kyber registered gitlab # [ 0.186732] pl061_gpio 9030000.pl061: PL061 GPIO chip registered gitlab # [ 0.190696] ledtrig-cpu: registered to indicate activity on CPUs gitlab # [ 0.191097] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: gitlab # [ 0.191112] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 gitlab # [ 0.191122] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 gitlab # [ 0.191128] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 gitlab # [ 0.191148] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits gitlab # [ 0.191171] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] gitlab # [ 0.191251] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 gitlab # [ 0.191259] pci_bus 0000:00: root bus resource [bus 00-ff] gitlab # [ 0.191263] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] gitlab # [ 0.191266] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] gitlab # [ 0.191270] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] gitlab # [ 0.191337] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint gitlab # [ 0.191904] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.192129] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.192142] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.192175] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.192188] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] gitlab # [ 0.192764] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.192992] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.193004] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.193036] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.193582] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.193806] pci 0000:00:03.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.193819] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.193850] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.194407] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.194645] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.194657] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.194688] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.195235] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.195452] pci 0000:00:05.0: BAR 0 [io 0x0000-0x007f] gitlab # [ 0.195464] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.195497] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.196033] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.196247] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f] gitlab # [ 0.196259] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.196291] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.196876] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.197098] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.197110] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.197138] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.197150] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref] gitlab # [ 0.197696] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint gitlab # [ 0.197923] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.197951] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.198537] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint gitlab # [ 0.198760] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.198789] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.199287] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint gitlab # [ 0.199517] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff] gitlab # [ 0.199805] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint gitlab # [ 0.200136] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] gitlab # [ 0.200154] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.200183] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.200759] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.200981] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.200993] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.201026] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.201722] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned gitlab # [ 0.201730] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned gitlab # [ 0.201733] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned gitlab # [ 0.201780] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned gitlab # [ 0.201832] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned gitlab # [ 0.201886] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned gitlab # [ 0.201938] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned gitlab # [ 0.201988] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned gitlab # [ 0.202035] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned gitlab # [ 0.202089] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned gitlab # [ 0.202141] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned gitlab # [ 0.202192] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned gitlab # [ 0.202257] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned gitlab # [ 0.202305] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned gitlab # [ 0.202324] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned gitlab # [ 0.202348] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned gitlab # [ 0.202372] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned gitlab # [ 0.202392] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned gitlab # [ 0.202411] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned gitlab # [ 0.202430] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned gitlab # [ 0.202450] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned gitlab # [ 0.202469] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned gitlab # [ 0.202511] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned gitlab # [ 0.202537] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned gitlab # [ 0.202562] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned gitlab # [ 0.202581] pci 0000:00:05.0: BAR 0 [io 0x1000-0x107f]: assigned gitlab # [ 0.202605] pci 0000:00:06.0: BAR 0 [io 0x1080-0x10ff]: assigned gitlab # [ 0.202623] pci 0000:00:0b.0: BAR 0 [io 0x1100-0x113f]: assigned gitlab # [ 0.202642] pci 0000:00:01.0: BAR 0 [io 0x1140-0x115f]: assigned gitlab # [ 0.202666] pci 0000:00:02.0: BAR 0 [io 0x1160-0x117f]: assigned gitlab # [ 0.202686] pci 0000:00:03.0: BAR 0 [io 0x1180-0x119f]: assigned gitlab # [ 0.202706] pci 0000:00:04.0: BAR 0 [io 0x11a0-0x11bf]: assigned gitlab # [ 0.202726] pci 0000:00:07.0: BAR 0 [io 0x11c0-0x11df]: assigned gitlab # [ 0.202746] pci 0000:00:0c.0: BAR 0 [io 0x11e0-0x11ff]: assigned gitlab # [ 0.202772] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] gitlab # [ 0.202777] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] gitlab # [ 0.202780] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] gitlab # [ 0.204288] pci 0000:00:0a.0: enabling device (0000 -> 0002) gitlab # [ 0.216717] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) gitlab # [ 0.219092] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) gitlab # [ 0.221358] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) gitlab # [ 0.223734] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) gitlab # [ 0.226063] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003) gitlab # [ 0.230814] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003) gitlab # [ 0.233023] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003) gitlab # [ 0.235378] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) gitlab # [ 0.237655] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) gitlab # [ 0.239773] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) gitlab # [ 0.244609] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) gitlab # [ 0.250402] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled gitlab # [ 0.252235] msm_serial: driver initialized gitlab # [ 0.252364] SuperH (H)SCI(F) driver initialized gitlab # [ 0.252412] STM32 USART driver initialized gitlab # [ 0.274194] loop: module loaded gitlab # [ 0.274454] virtio_blk virtio4: 4/0/0 default/read/poll queues gitlab # [ 0.276411] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) gitlab # [ 0.278518] virtio_blk virtio5: 4/0/0 default/read/poll queues gitlab # [ 0.280452] virtio_blk virtio5: [vdb] 12285632 512-byte logical blocks (6.29 GB/5.86 GiB) gitlab # [ 0.282543] megasas: 07.734.00.00-rc1 gitlab # [ 0.283270] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] gitlab # [ 0.288053] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 gitlab # [ 0.288074] Intel/Sharp Extended Query Table at 0x0031 gitlab # [ 0.289748] Using buffer write method gitlab # [ 0.289784] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] gitlab # [ 0.291630] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 gitlab # [ 0.291648] Intel/Sharp Extended Query Table at 0x0031 gitlab # [ 0.293336] Using buffer write method gitlab # [ 0.293354] Concatenating MTD devices: gitlab # [ 0.293356] (0): "0.flash" gitlab # [ 0.293359] (1): "0.flash" gitlab # [ 0.293361] into device "0.flash" gitlab # [ 0.628812] Freeing initrd memory: 27184K gitlab # [ 0.639607] tun: Universal TUN/TAP device driver, 1.6 gitlab # [ 0.645148] thunder_xcv, ver 1.0 gitlab # [ 0.645237] thunder_bgx, ver 1.0 gitlab # [ 0.645267] nicpf, ver 1.0 gitlab # [ 0.646179] e1000: Intel(R) PRO/1000 Network Driver gitlab # [ 0.646186] e1000: Copyright (c) 1999-2006 Intel Corporation. gitlab # [ 0.646221] e1000e: Intel(R) PRO/1000 Network Driver gitlab # [ 0.646229] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. gitlab # [ 0.646282] igb: Intel(R) Gigabit Ethernet Network Driver gitlab # [ 0.646287] igb: Copyright (c) 2007-2014 Intel Corporation. gitlab # [ 0.646320] igbvf: Intel(R) Gigabit Virtual Function Network Driver gitlab # [ 0.646325] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. gitlab # [ 0.646590] sky2: driver version 1.30 gitlab # [ 0.649071] usbcore: registered new interface driver usb-storage gitlab # [ 0.649314] usbcore: registered new interface driver usbserial_generic gitlab # [ 0.649335] usbserial: USB Serial support registered for generic gitlab # [ 0.649391] ehci-pci 0000:00:0a.0: EHCI Host Controller gitlab # [ 0.649430] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1 gitlab # [ 0.649751] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000 gitlab # [ 0.650281] hv_vmbus: registering driver hyperv_keyboard gitlab # [ 0.652070] rtc-pl031 9010000.pl031: registered as rtc0 gitlab # [ 0.652112] rtc-pl031 9010000.pl031: setting system clock to 2026-10-02T16:17:23 UTC (1790957843) gitlab # [ 0.652611] i2c_dev: i2c /dev entries driver gitlab # [ 0.657054] sdhci: Secure Digital Host Controller Interface driver gitlab # [ 0.657065] sdhci: Copyright(c) Pierre Ossman gitlab # [ 0.657491] Synopsys Designware Multimedia Card Interface Driver gitlab # [ 0.658055] sdhci-pltfm: SDHCI platform and OF driver helper gitlab # [ 0.658550] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00 gitlab # [ 0.659189] hub 1-0:1.0: USB hub found gitlab # [ 0.659229] hub 1-0:1.0: 6 ports detected gitlab # [ 0.660679] hid: raw HID events driver (C) Jiri Kosina gitlab # [ 0.661074] usbcore: registered new interface driver usbhid gitlab # [ 0.661080] usbhid: USB HID core driver gitlab # [ 0.695052] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available gitlab # [ 0.697453] drop_monitor: Initializing network drop monitor service gitlab # [ 0.697718] NET: Registered PF_INET6 protocol family gitlab # [ 0.699209] Segment Routing with IPv6 gitlab # [ 0.699236] In-situ OAM (IOAM) with IPv6 gitlab # [ 0.699279] NET: Registered PF_PACKET protocol family gitlab # [ 0.699562] 9pnet: Installing 9P2000 support gitlab # [ 0.702236] Key type dns_resolver registered gitlab # [ 0.711886] registered taskstats version 1 gitlab # [ 0.712304] Loading compiled-in X.509 certificates gitlab # [ 0.724537] Demotion targets for Node 0: null gitlab # [ 0.724780] Key type .fscrypt registered gitlab # [ 0.724786] Key type fscrypt-provisioning registered gitlab # [ 0.724945] ima: No TPM chip found, activating TPM-bypass! gitlab # [ 0.724967] ima: Allocated hash algorithm: sha1 gitlab # [ 0.724993] ima: No architecture policies found gitlab # [ 0.725984] input: gpio-keys as /devices/platform/gpio-keys/input/input0 gitlab # [ 0.749017] clk: Disabling unused clocks gitlab # [ 0.749055] PM: genpd: Disabling unused power domains gitlab # [ 0.753114] Freeing unused kernel memory: 4736K gitlab # [ 0.753325] Run /init as init process gitlab # [ 0.782302] systemd[1]: Successfully made /usr/ read-only. gitlab # [ 0.918541] usb 1-1: new high-speed USB device number 2 using ehci-pci gitlab # [ 1.068191] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 gitlab # [ 1.116554] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 1.116594] systemd[1]: Detected virtualization qemu. gitlab # [ 1.116639] systemd[1]: Detected architecture arm64. gitlab # [ 1.116648] systemd[1]: Running in initrd. gitlab # [ 1.117173] systemd[1]: Initializing machine ID from random generator. gitlab # [ 1.117325] systemd[1]: Hostname set to . gitlab # [ 1.186901] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0 gitlab # [ 1.306517] usb 1-2: new high-speed USB device number 3 using ehci-pci gitlab # [ 1.322020] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 1.384055] systemd[1]: Queued start job for default target Initrd Default Target. gitlab # [ 1.417861] systemd[1]: Created slice Slice /system/modprobe. gitlab # [ 1.418167] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 1.418208] systemd[1]: Expecting device /dev/disk/by-label/nix-store... gitlab # [ 1.418223] systemd[1]: Expecting device /dev/disk/by-label/nixos... gitlab # [ 1.418248] systemd[1]: Reached target Path Units. gitlab # [ 1.418267] systemd[1]: Reached target Slice Units. gitlab # [ 1.418286] systemd[1]: Reached target Swaps. gitlab # [ 1.418304] systemd[1]: Reached target Timer Units. gitlab # [ 1.418666] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 1.418871] systemd[1]: Listening on Journal Socket (/dev/log). gitlab # [ 1.419059] systemd[1]: Listening on Journal Sockets. gitlab # [ 1.419238] systemd[1]: Listening on udev Control Socket. gitlab # [ 1.419350] systemd[1]: Listening on udev Kernel Socket. gitlab # [ 1.419371] systemd[1]: Reached target Socket Units. gitlab # [ 1.421520] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 1.423572] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 1.423661] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs gitlab # [ 1.425876] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 1.441398] systemd[1]: Starting Journal Service... gitlab # [ 1.443691] systemd[1]: Starting Load Kernel Modules... gitlab # [ 1.443860] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 1.449438] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 1.454417] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 1.456016] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 1.456569] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 1.456984] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 1.460779] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 1.462280] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 gitlab # [ 1.462559] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0 gitlab # [ 1.480836] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0 gitlab # [ 1.481089] [drm] features: -virgl +edid -resource_blob -host_visible gitlab # [ 1.481094] [drm] features: -context_init gitlab # [ 1.482408] [drm] number of scanouts: 1 gitlab # [ 1.482434] [drm] number of cap sets: 0 gitlab # [ 1.482930] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 1.488822] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. gitlab # [ 1.494855] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 1.494992] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic gitlab # [ 1.495011] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0 gitlab # [ 1.495209] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev gitlab # [ 1.498470] systemd-journald[95]: Collecting audit messages is disabled. gitlab # [ 1.506082] Console: switching to colour frame buffer device 160x50 gitlab # [ 1.510776] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device gitlab # [ 1.528862] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 1.529664] systemd[1]: Finished Load Kernel Modules. gitlab # [ 1.529839] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 1.529885] systemd[1]: Reached target Local File Systems. gitlab # [ 1.531471] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 1.535331] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 1.549445] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 1.590836] systemd[1]: Started Journal Service. gitlab # [ 1.588443] systemd-modules-load[96]: Using 4 probe threads gitlab # [ 1.591576] systemd-modules-load[96]: Module 'virtio_balloon' is built in gitlab # [ 1.593583] systemd-modules-load[96]: Module 'virtio_console' is built in gitlab # [ 1.595365] systemd-modules-load[96]: Module 'virtio_rng' is built in gitlab # [ 1.598109] systemd-modules-load[96]: Inserted module 'dm_mod' gitlab # [ 1.600447] systemd-modules-load[96]: Inserted module 'virtio_gpu' gitlab # [ 1.603850] systemd-udevd[109]: Using default interface naming scheme 'v261'. gitlab # [ 1.607263] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 1.609227] systemd[1]: Starting Create System Files and Directories... gitlab # [ 1.623649] systemd[1]: Starting Virtual Console Setup... gitlab # [ 1.628659] systemd[1]: Finished Create System Files and Directories. gitlab # [ 1.643995] systemd-vconsole-setup[132]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 1.646543] systemd[1]: Finished Virtual Console Setup. gitlab # [ 2.098649] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 2.099822] systemd[1]: Reached target System Initialization. gitlab # [ 2.100825] systemd[1]: Reached target Basic System. gitlab # [ 2.243102] (udev-worker)[147]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 2.247340] (udev-worker)[147]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.288108] systemd[1]: Found device /dev/disk/by-label/nix-store. gitlab # [ 2.292541] systemd[1]: Found device /dev/disk/by-label/nixos. gitlab # [ 2.294136] systemd[1]: Reached target Initrd Root Device. gitlab # [ 2.295970] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... gitlab # [ 2.298123] (udev-worker)[125]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.341113] systemd-fsck[155]: nixos: clean, 12/65536 files, 13019/262144 blocks gitlab # [ 2.347865] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. gitlab # [ 2.450094] systemd[1]: Mounting /sysroot... gitlab # [ 2.487000] EXT4-fs (vda): mounted filesystem 3c54de56-811b-4570-9652-a6366a149d56 r/w with ordered data mode. Quota mode: none. gitlab # [ 2.487412] systemd[1]: Mounted /sysroot. gitlab # [ 2.488419] systemd[1]: Reached target Initrd Root File System. gitlab # [ 2.492195] systemd[1]: Mounting /sysroot/nix/.ro-store... gitlab # [ 2.493891] systemd[1]: Mounting /sysroot/nix/.rw-store... gitlab # [ 2.496211] systemd[1]: Mounting /sysroot/run... gitlab # [ 2.501835] systemd[1]: Mounting /sysroot/tmp/shared... gitlab # [ 2.510277] systemd[1]: Mounting /sysroot/tmp/xchg... gitlab # [ 2.512534] systemd[1]: Starting Mountpoints Configured in the Real Root... gitlab # [ 2.530650] systemd[1]: Mounted /sysroot/nix/.rw-store. gitlab # [ 2.533002] systemd-sysroot-fstab-check[168]: /sysroot should be mounted in the initrd, will request daemon-reload. gitlab # [ 2.536123] systemd[1]: Mounted /sysroot/run. gitlab # [ 2.544793] systemd[1]: Mounted /sysroot/tmp/xchg. gitlab # [ 2.548140] systemd[1]: Reload requested from client PID 168 ('systemd-sysroot') (unit initrd-parse-etc.service)... gitlab # [ 2.549902] systemd[1]: Reloading... gitlab # [ 2.552510] erofs (device vdb): mounted with root inode @ nid 128. gitlab # [ 2.659694] systemd[1]: Reloading finished in 112 ms. gitlab # [ 2.712731] systemd-sysroot-fstab-check[168]: Requesting initrd-fs.target/start/replace... gitlab # [ 2.714172] systemd[1]: Mounted /sysroot/nix/.ro-store. gitlab # [ 2.715351] systemd[1]: Mounted /sysroot/tmp/shared. gitlab # [ 2.717252] systemd-sysroot-fstab-check[168]: Requesting swap.target/start/replace... gitlab # [ 2.719612] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 2.721476] systemd[1]: initrd-parse-etc.service: Deactivated successfully. gitlab # [ 2.723628] systemd[1]: Finished Mountpoints Configured in the Real Root. gitlab # [ 2.725265] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. gitlab # [ 2.743824] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 2.745863] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 2.813419] (udev-worker)[147]: mtd0ro: Failed to find and pin callout binary "/nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/mtd_probe": No such file or directory gitlab # [ 2.817350] (udev-worker)[147]: mtd0ro: /nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory gitlab # [ 2.824691] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 2.826193] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 2.827059] systemd[1]: Stopping Virtual Console Setup... gitlab # [ 2.828130] systemd[1]: Starting Virtual Console Setup... gitlab # [ 2.859535] systemd-vconsole-setup[198]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 2.861656] systemd[1]: Finished Virtual Console Setup. gitlab # [ 3.456787] systemd[1]: Mounting /sysroot/nix/store... gitlab # [ 3.506797] systemd[1]: Mounted /sysroot/nix/store. gitlab # [ 3.508243] systemd[1]: Reached target Initrd File Systems. gitlab # [ 3.510131] systemd[1]: Starting Find NixOS closure... gitlab # [ 3.512288] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... gitlab # [ 3.548160] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. gitlab # [ 3.550140] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. gitlab # [ 3.563345] systemd[1]: Finished Find NixOS closure. gitlab # [ 3.564801] systemd[1]: Reached target Initrd Default Target. gitlab # [ 3.566120] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... gitlab # [ 3.600860] systemd[1]: Stopped target Initrd Default Target. gitlab # [ 3.602259] systemd[1]: Stopped target Basic System. gitlab # [ 3.603423] systemd[1]: Stopped target Initrd Root Device. gitlab # [ 3.604765] systemd[1]: Stopped target Path Units. gitlab # [ 3.605896] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. gitlab # [ 3.607580] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. gitlab # [ 3.609394] systemd[1]: Stopped target Slice Units. gitlab # [ 3.610549] systemd[1]: Stopped target Socket Units. gitlab # [ 3.612293] systemd[1]: Stopped target System Initialization. gitlab # [ 3.614823] systemd[1]: Stopped target Swaps. gitlab # [ 3.616999] systemd[1]: Stopped target Timer Units. gitlab # [ 3.619073] systemd[1]: dbus.socket: Deactivated successfully. gitlab # [ 3.622113] systemd[1]: Closed D-Bus System Message Bus Socket. gitlab # [ 3.624420] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. gitlab # [ 3.627310] systemd[1]: Stopped Find NixOS closure. gitlab # [ 3.628891] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 3.630451] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.631831] systemd[1]: systemd-sysctl.service: Deactivated successfully. gitlab # [ 3.633646] systemd[1]: Stopped Apply Kernel Variables. gitlab # [ 3.634957] systemd[1]: systemd-modules-load.service: Deactivated successfully. gitlab # [ 3.637123] systemd[1]: Stopped Load Kernel Modules. gitlab # [ 3.638369] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. gitlab # [ 3.640211] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. gitlab # [ 3.642031] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. gitlab # [ 3.643694] systemd[1]: Stopped Create System Files and Directories. gitlab # [ 3.645344] systemd[1]: Stopped target Local File Systems. gitlab # [ 3.646606] systemd[1]: Stopped target Preparation for Local File Systems. gitlab # [ 3.648146] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. gitlab # [ 3.649737] systemd[1]: Stopped Coldplug All udev Devices. gitlab # [ 3.650980] systemd[1]: Stopping Rule-based Manager for Device Events and Files... gitlab # [ 3.652719] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 3.654359] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 3.655542] systemd[1]: initrd-cleanup.service: Deactivated successfully. gitlab # [ 3.657104] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. gitlab # [ 3.658500] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 3.660166] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 3.661456] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.662912] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.664146] systemd[1]: systemd-udevd.service: Deactivated successfully. gitlab # [ 3.665479] systemd[1]: Stopped Rule-based Manager for Device Events and Files. gitlab # [ 3.666912] systemd[1]: systemd-udevd.service: Consumed 2.093s CPU time over 2.110s wall clock time, 32M memory peak. gitlab # [ 3.669201] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. gitlab # [ 3.670576] systemd[1]: Closed udev Control Socket. gitlab # [ 3.671539] systemd[1]: Starting Cleanup udev Database... gitlab # [ 3.672651] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. gitlab # [ 3.674045] systemd[1]: Stopped Create Static Device Nodes in /dev. gitlab # [ 3.675188] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. gitlab # [ 3.676714] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. gitlab # [ 3.677925] systemd[1]: kmod-static-nodes.service: Deactivated successfully. gitlab # [ 3.679111] systemd[1]: Stopped Create List of Static Device Nodes. gitlab # [ 3.719438] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. gitlab # [ 3.721279] systemd[1]: Finished Cleanup udev Database. gitlab # [ 3.722466] systemd[1]: Reached target Switch Root. gitlab # [ 3.723584] systemd[1]: Starting NixOS Activation... gitlab # [ 3.848036] initrd-nixos-activation-start[224]: booting system configuration /nix/store/1lnw6hp2c14n58kyqwr96rnmq7dapmi1-nixos-system-gitlab-test gitlab # [ 3.887559] initrd-nixos-activation-start[224]: running activation script... gitlab # [ 4.129940] initrd-nixos-activation-start[247]: setting up /etc... gitlab # [ 4.242662] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. gitlab # [ 4.244652] systemd[1]: Finished NixOS Activation. gitlab # [ 4.245793] systemd[1]: Starting Switch Root... gitlab # [ 4.280245] systemd[1]: Switching root. gitlab # [ 4.418157] systemd-journald[95]: Received SIGTERM from PID 1 (systemd). gitlab # [ 4.984509] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 4.984592] systemd[1]: Detected virtualization qemu. gitlab # [ 4.984658] systemd[1]: Detected architecture arm64. gitlab # [ 4.984782] systemd[1]: Detected first boot. gitlab # [ 4.985667] systemd[1]: Initializing machine ID from random generator. gitlab # [ 5.189869] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 5.281067] systemd[1]: Applying preset policy. gitlab # [ 5.453338] systemd[1]: Populated /etc with preset unit settings. gitlab # [ 5.622249] systemd[1]: initrd-switch-root.service: Deactivated successfully. gitlab # [ 5.622626] systemd[1]: Stopped initrd-switch-root.service. gitlab # [ 5.624365] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. gitlab # [ 5.627195] systemd[1]: Created slice Slice /system/getty. gitlab # [ 5.629645] systemd[1]: Created slice GitLab DevOps Platform Slice. gitlab # [ 5.631949] systemd[1]: Created slice User and Session Slice. gitlab # [ 5.632515] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 5.633107] systemd[1]: Started Forward Password Requests to Wall Directory Watch. gitlab # [ 5.633642] systemd[1]: Expecting device /dev/hvc0... gitlab # [ 5.634194] systemd[1]: Expecting device /dev/ttyAMA0... gitlab # [ 5.637580] systemd[1]: Reached target Local Encrypted Volumes. gitlab # [ 5.637966] systemd[1]: Stopped target initrd-fs.target. gitlab # [ 5.638545] systemd[1]: Stopped target initrd-root-fs.target. gitlab # [ 5.639068] systemd[1]: Stopped target initrd-switch-root.target. gitlab # [ 5.639632] systemd[1]: Reached target Virtual Machines and Containers. gitlab # [ 5.640181] systemd[1]: Reached target Path Units. gitlab # [ 5.640705] systemd[1]: Reached target Remote File Systems. gitlab # [ 5.641236] systemd[1]: Reached target Slice Units. gitlab # [ 5.641783] systemd[1]: Reached target Swaps. gitlab # [ 5.643552] systemd[1]: Listening on Query the User Interactively for a Password. gitlab # [ 5.645226] systemd[1]: Listening on Process Core Dump Socket. gitlab # [ 5.646404] systemd[1]: Listening on Credential Encryption/Decryption. gitlab # [ 5.652201] systemd[1]: Listening on Factory Reset Management. gitlab # [ 5.652840] systemd[1]: Listening on Hostname Service Socket. gitlab # [ 5.657953] systemd[1]: Starting Journal Log Access Socket... gitlab # [ 5.659446] systemd[1]: Listening on Journal Audit Socket. gitlab # [ 5.660971] systemd[1]: Listening on Console Output Muting Service Socket. gitlab # [ 5.661450] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. gitlab # [ 5.661738] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.661974] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 5.664335] systemd[1]: Listening on Disk Repartitioning Service Socket. gitlab # [ 5.664888] systemd[1]: Listening on udev Control Socket. gitlab # [ 5.665528] systemd[1]: Listening on udev Varlink Socket. gitlab # [ 5.669965] systemd[1]: Mounting Huge Pages File System... gitlab # [ 5.674060] systemd[1]: Mounting POSIX Message Queue File System... gitlab # [ 5.681682] systemd[1]: Mounting Kernel Debug File System... gitlab # [ 5.687751] systemd[1]: Mounting Kernel Trace File System... gitlab # [ 5.691598] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 5.700955] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 5.702975] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs gitlab # [ 5.710658] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 5.711278] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm gitlab # [ 5.711935] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 5.722631] systemd[1]: Starting Load Kernel Module fuse... gitlab # [ 5.723904] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 5.729637] systemd[1]: Starting Journal Service... gitlab # [ 5.732191] systemd[1]: Starting Load Kernel Modules... gitlab # [ 5.736252] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... gitlab # [ 5.745582] systemd[1]: Starting Remount Root and Kernel File Systems... gitlab # [ 5.746835] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.751360] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 5.752004] fuse: init (API version 7.45) gitlab # [ 5.757571] systemd[1]: Listening on Journal Log Access Socket. gitlab # [ 5.761993] systemd[1]: Mounted Huge Pages File System. gitlab # [ 5.764477] systemd[1]: Mounted POSIX Message Queue File System. gitlab # [ 5.766418] systemd[1]: Mounted Kernel Debug File System. gitlab # [ 5.768905] systemd[1]: Mounted Kernel Trace File System. gitlab # [ 5.772849] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 5.773928] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 5.775248] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 5.776148] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 5.777058] systemd[1]: modprobe@fuse.service: Deactivated successfully. gitlab # [ 5.777385] systemd-journald[318]: Collecting audit messages is enabled. gitlab # [ 5.780580] systemd[1]: Finished Load Kernel Module fuse. gitlab # [ 5.781895] systemd[1]: Finished Load Kernel Modules. gitlab # [ 5.786983] systemd[1]: Mounting FUSE Control File System... gitlab # [ 5.796489] systemd[1]: Starting Firewall... gitlab # [ 5.808430] systemd[1]: Queued start job for default target Multi-User System. gitlab # [ 5.810017] systemd[1]: Unnecessary job was removed for /dev/vdb. gitlab # [ 5.811301] systemd[1]: systemd-journald.service: Deactivated successfully.[ 5.814730] systemd[1]: Starting Apply Kernel Variables... gitlab # gitlab # [ 5.819381] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 5.819666] systemd-modules-load[319]: Using 3 probe threads[ 5.821565] EXT4-fs (vda): re-mounted 3c54de56-811b-4570-9652-a6366a149d56. gitlab # [ 5.821735] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. gitlab # gitlab # [ 5.825533] systemd[1]: Started Journal Service. gitlab # [ 5.836359] systemd-modules-load[319]: Module 'atkbd' is built in gitlab # [ 5.839005] systemd-modules-load[319]: Module 'loop' is built in gitlab # [ 5.840770] systemd-modules-load[319]: Inserted module 'tls' gitlab # [ 5.842418] systemd-oomd[320]: No swap; memory pressure usage will be degraded gitlab # [ 5.845002] systemd[1]: Mounted FUSE Control File System. gitlab # [ 5.846102] systemd[1]: Finished Remount Root and Kernel File Systems. gitlab # [ 5.847209] systemd[1]: Listening on Disk Image Download Service Socket. gitlab # [ 5.851202] systemd[1]: Starting Flush Journal to Persistent Storage... gitlab # [ 5.856534] systemd-journald[318]: Received client request to flush runtime journal. gitlab # [ 5.960867] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 5.963470] systemd[1]: Starting Load/Save OS Random Seed... gitlab # [ 5.966247] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.969936] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 5.971491] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 5.973218] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 5.977027] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 5.982203] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 5.983987] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 5.986680] systemd[1]: Finished Load/Save OS Random Seed. gitlab # [ 5.988296] systemd[1]: Reached target First Boot Complete. gitlab # [ 5.989398] systemd[1]: Finished Flush Journal to Persistent Storage. gitlab # [ 6.032731] systemd-udevd[367]: Using default interface naming scheme 'v261'. gitlab # [ 6.155404] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 6.381737] systemd[1]: Finished Firewall. gitlab # [ 6.614511] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 6.623730] systemd[1]: Mounting /run/wrappers... gitlab # [ 6.653897] systemd[1]: Mounted /run/wrappers. gitlab # [ 6.655083] systemd[1]: Reached target Local File Systems. gitlab # [ 6.656630] systemd[1]: Listening on Boot Loader Control Service Socket. gitlab # [ 6.659361] systemd[1]: Starting register-nix-paths.service... gitlab # [ 6.662486] systemd[1]: Starting Create SUID/SGID Wrappers... gitlab # [ 6.666191] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 6.669604] systemd[1]: Starting Save Transient machine-id to Disk... gitlab # [ 6.680228] systemd[1]: Starting Create System Files and Directories... gitlab # [ 6.693843] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs gitlab # [ 6.758208] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. gitlab # [ 6.760345] systemd[1]: Finished Save Transient machine-id to Disk. gitlab # [ 6.765204] systemd[1]: Finished Create System Files and Directories. gitlab # [ 6.776233] systemd[1]: Starting Rebuild Journal Catalog... gitlab # [ 6.781189] systemd[1]: Starting Record System Boot/Shutdown in UTMP... gitlab # [ 6.784129] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. gitlab # [ 6.823431] systemd[1]: Finished Record System Boot/Shutdown in UTMP. gitlab # [ 6.831568] (udev-worker)[428]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.841044] systemd[1]: Finished Rebuild Journal Catalog. gitlab # [ 6.845556] systemd[1]: Starting Update is Completed... gitlab # [ 6.855245] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. gitlab # [ 6.859432] (udev-worker)[420]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 6.861828] (udev-worker)[420]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.873484] systemd[1]: Finished Update is Completed. gitlab # [ 6.888452] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. gitlab # [ 6.890177] systemd[1]: Finished Create SUID/SGID Wrappers. gitlab # [ 6.891127] systemd[1]: suid-sgid-wrappers.service: Consumed 150ms CPU time over 227ms wall clock time, 6.1M memory peak, 1.9M read from disk. gitlab # [ 6.897003] systemd[1]: Condition check resulted in Virtio network device being skipped. gitlab # [ 6.898301] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 6.899908] systemd[1]: Starting Create SUID/SGID Wrappers... gitlab # [ 6.901079] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 6.902787] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 6.905698] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 6.907532] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.909178] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 7.003304] mousedev: PS/2 mouse device common for all mice gitlab # [ 7.081546] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. gitlab # [ 7.084332] systemd[1]: Finished Create SUID/SGID Wrappers. gitlab # [ 7.157755] systemd[1]: Finished register-nix-paths.service. gitlab # [ 7.159386] systemd[1]: Reached target System Initialization. gitlab # [ 7.160578] systemd[1]: Started Discard unused filesystem blocks once a week. gitlab # [ 7.162308] systemd[1]: Started logrotate.timer. gitlab # [ 7.164332] systemd[1]: Started Daily Cleanup of Temporary Directories. gitlab # [ 7.165339] systemd[1]: Reached target Timer Units. gitlab # [ 7.166102] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 7.167140] systemd[1]: Listening on Nix Daemon Socket. gitlab # [ 7.177368] systemd[1]: Starting nixbot CI service socket... gitlab # [ 7.178840] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. gitlab # [ 7.181381] systemd[1]: Starting D-Bus System Message Bus... gitlab # [ 7.182383] systemd[1]: Listening on nixbot CI service socket. gitlab # [ 7.184200] systemd[1]: Reached target Socket Units. gitlab # [ 7.220608] dbus-broker-launch[619]: Looking up NSS user entry for 'systemd-timesync'... gitlab # [ 7.223394] dbus-broker-launch[619]: NSS returned no entry for 'systemd-timesync' gitlab # [ 7.225608] dbus-broker-launch[619]: Invalid user-name in /nix/store/9zcyylhndlnizbwn48627991zr1jxg4p-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" gitlab # [ 7.229204] systemd[1]: Started D-Bus System Message Bus. gitlab # [ 7.238162] dbus-broker-launch[619]: Ready gitlab # [ 7.239604] systemd[1]: Reached target Basic System. gitlab # [ 7.241599] systemd[1]: Started backdoor.service. gitlab # [ 7.247693] systemd[1]: Starting gitlab-config.service... gitlab # [ 7.260200] systemd[1]: Starting Import lastlog data into lastlog2 database... gitlab # [ 7.268143] systemd[1]: Starting Logrotate configuration check... gitlab # [ 7.280719] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.283221] logrotate[624]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. gitlab # connecting to host... gitlab # [ 7.286864] logrotate[624]: reading config file /nix/store/22afkmbkaxyx50bj9lbkkpmz7pd69hsf-logrotate.conf gitlab # [ 7.293853] logrotate[624]: note: 'monthly' overrides previously specified 'weekly' gitlab # [ 7.295102] logrotate[624]: Reading state from file: /var/lib/logrotate.status gitlab: Guest shell says: b'Spawning backdoor root shell...\n' gitlab: connected to guest root shell gitlab # [ 7.308693] logrotate[624]: state file /var/lib/logrotate.status does not exist gitlab: (connecting took 7.77 seconds) gitlab # [ 7.310551] logrotate[624]: Allocating hash table for state file, size 64 entries gitlab: (finished: waiting for the VM to finish booting, in 7.77 seconds) gitlab # [ 7.313452] logrotate[624]: Handling 4 logs gitlab # [ 7.314115] logrotate[624]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.318538] logrotate[624]: considering log /var/log/btmp gitlab # [ 7.319282] logrotate[624]: Creating new state gitlab # [ 7.320045] logrotate[624]: Now: 2026-10-02 16:17 gitlab # [ 7.320736] logrotate[624]: Last rotated at 2026-10-02 16:00 gitlab # [ 7.321534] logrotate[624]: log does not need rotating (log has already been rotated) gitlab # [ 7.322587] logrotate[624]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.325060] logrotate[624]: considering log /var/log/wtmp gitlab # [ 7.328763] logrotate[624]: Creating new state gitlab # [ 7.329518] logrotate[624]: Now: 2026-10-02 16:17 gitlab # [ 7.330230] logrotate[624]: Last rotated at 2026-10-02 16:00 gitlab # [ 7.331023] logrotate[624]: log does not need rotating (log has already been rotated) gitlab # [ 7.332876] logrotate[624]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed gitlab # [ 7.339738] logrotate[624]: switching euid from 0 to 165 and egid from 0 to 165 (pid 624) gitlab # [ 7.341821] logrotate[624]: considering log /var/gitlab/state/log/*.log gitlab # [ 7.342773] logrotate[624]: log /var/gitlab/state/log/*.log does not exist -- skipping gitlab # [ 7.343876] logrotate[624]: Creating new state gitlab # [ 7.344711] logrotate[624]: switching euid from 165 to 0 and egid from 165 to 0 (pid 624) gitlab # [ 7.347309] logrotate[624]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed gitlab # [ 7.353345] logrotate[624]: switching euid from 0 to 60 and egid from 0 to 60 (pid 624) gitlab # [ 7.354508] logrotate[624]: considering log /var/log/nginx/*.log gitlab # [ 7.355361] logrotate[624]: log /var/log/nginx/*.log does not exist -- skipping gitlab # [ 7.356572] logrotate[624]: Creating new state gitlab # [ 7.357229] logrotate[624]: not running postrotate script, since no logs were rotated gitlab # [ 7.358273] logrotate[624]: switching euid from 60 to 0 and egid from 60 to 0 (pid 624) gitlab # [ 7.359397] systemd[1]: Starting Post-Boot Actions... gitlab # [ 7.361723] systemd[1]: Started Reset console on configuration changes. gitlab # [ 7.362890] systemd[1]: Starting resolvconf update... gitlab # [ 7.364139] nsncd[629]: Oct 02 16:17:30.200 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.366191] systemd[1]: Finished Logrotate configuration check. gitlab # [ 7.368446] systemd[1]: Finished Post-Boot Actions. gitlab # [ 7.369471] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.370467] systemd[1]: Finished Import lastlog data into lastlog2 database. gitlab # [ 7.371885] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.374300] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.375302] systemd[1]: Starting User Login Management... gitlab # [ 7.431427] systemd-logind[658]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) gitlab # [ 7.433840] systemd-logind[658]: Watching system buttons on /dev/input/event0 (gpio-keys) gitlab # [ 7.467644] systemd[1]: Stopped target Host and Network Name Lookups. gitlab # [ 7.531818] systemd[1]: Stopping Host and Network Name Lookups... gitlab # [ 7.669205] systemd[1]: Stopped target User and Group Name Lookups. gitlab # [ 7.670475] nsncd[725]: Oct 02 16:17:30.394 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.672756] systemd[1]: Stopping User and Group Name Lookups... gitlab # [ 7.673987] network-addresses-eth1-start[740]: adding address 192.168.1.1/24... done gitlab # [ 7.675084] network-addresses-eth1-start[740]: adding address 2001:db8:1::1/64... done gitlab # [ 7.678641] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... gitlab # [ 7.701751] systemd[1]: nscd.service: Deactivated successfully. gitlab # [ 7.702627] systemd-logind[658]: New seat seat0. gitlab # [ 7.703346] systemd[1]: Stopped Name Service Cache Daemon (nsncd). gitlab # [ 7.704287] systemd[1]: nscd.service: Consumed 33ms CPU time over 188ms wall clock time, 7.5M memory peak, 5.8M read from disk. gitlab # [ 7.705872] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.706770] systemd[1]: Finished resolvconf update. gitlab # [ 7.708243] systemd[1]: Reached target Preparation for Network. gitlab # [ 7.709540] systemd[1]: Starting DHCP Client... gitlab # [ 7.710261] systemd[1]: Starting Address configuration of eth1... gitlab # [ 7.711281] systemd[1]: Starting Extra networking commands.... gitlab # [ 7.712436] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.732733] systemd[1]: Finished Address configuration of eth1. gitlab # [ 7.733689] systemd[1]: Started User Login Management. gitlab # [ 7.737371] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.738468] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.761849] systemd[1]: Starting linger-users.service... gitlab # [ 7.770307] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3 gitlab # [ 7.803405] systemd[1]: linger-users.service: Deactivated successfully. gitlab # [ 7.806445] systemd[1]: Finished linger-users.service. gitlab # [ 7.819596] systemd[1]: Finished Extra networking commands.. gitlab # [ 7.823202] systemd[1]: Reached target Network. gitlab # [ 7.827185] systemd[1]: Starting gitlab-workhorse.service... gitlab # [ 7.847676] systemd[1]: Starting Nginx Web Server... gitlab # [ 7.864144] systemd[1]: Starting PostgreSQL Server... gitlab # [ 7.879668] systemd[1]: Starting Redis Server - redis-gitlab... gitlab # [ 7.881312] systemd[1]: Starting Permit User Sessions... gitlab # [ 7.920603] systemd[1]: Starting Virtual Console Setup... gitlab # [ 7.930651] systemd-logind[658]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) gitlab # [ 7.932938] systemd[1]: Finished Permit User Sessions. gitlab # [ 7.944175] systemd[1]: Started Getty on tty1. gitlab # [ 7.946784] systemd[1]: Reached target Login Prompts. gitlab # [ 7.997466] dhcpcd[804]: dhcpcd-10.3.2 starting gitlab # [ 8.002099] dhcpcd[839]: dev: loaded udev gitlab # [ 8.016814] 8021q: 802.1Q VLAN Support v1.8 gitlab # [ 8.017218] 8021q: adding VLAN 0 to HW filter on device eth1 gitlab # [ 8.028621] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. gitlab # [ 8.030407] systemd-vconsole-setup[818]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 8.035138] systemd[1]: Finished Virtual Console Setup. gitlab # [ 8.073247] nginx-pre-start[835]: nginx: the configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf syntax is ok gitlab # [ 8.077418] cfg80211: Loading compiled-in X.509 certificates for regulatory database gitlab # [ 8.077714] nginx-pre-start[835]: nginx: configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf test is successful gitlab # [ 8.084854] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' gitlab # [ 8.085353] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' gitlab # [ 8.086673] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 gitlab # [ 8.083310] systemd[1]: Started Nginx Web Server.[ 8.086978] cfg80211: failed to load regulatory.db gitlab # gitlab # [ 8.114805] redis-server[833]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. gitlab # [ 8.117914] redis-server[833]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo gitlab # [ 8.119840] redis-server[833]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=833, just started gitlab # [ 8.122867] redis-server[833]: Configuration loaded gitlab # [ 8.124439] redis[833]: monotonic clock: ARM CNTVCT @ 1000 ticks/us gitlab # [ 8.125538] redis[833]: Running mode=standalone, port=0. gitlab # [ 8.126779] systemd[1]: Started Redis Server - redis-gitlab. gitlab # [ 8.127884] redis[833]: Server initialized gitlab # [ 8.129265] redis[833]: Ready to accept connections unix gitlab # [ 8.133206] 8021q: adding VLAN 0 to HW filter on device eth0 gitlab # [ 8.131109] redis[833]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. gitlab # [ 8.134848] dhcpcd[839]: eth0: waiting for carrier gitlab # [ 8.136047] postgresql-pre-start[837]: The files belonging to this database system will be owned by user "postgres". gitlab # [ 8.137572] postgresql-pre-start[837]: This user must also own the server process. gitlab # [ 8.138753] postgresql-pre-start[837]: The database cluster will be initialized with locale "en_US.UTF-8". gitlab # [ 8.140271] postgresql-pre-start[837]: The default database encoding has accordingly been set to "UTF8". gitlab # [ 8.141615] postgresql-pre-start[837]: The default text search configuration will be set to "english". gitlab # [ 8.142947] postgresql-pre-start[837]: Data page checksums are enabled. gitlab # [ 8.143857] postgresql-pre-start[837]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok gitlab # [ 8.145447] dhcpcd[839]: eth0: carrier acquired gitlab # [ 8.146206] postgresql-pre-start[837]: creating subdirectories ... ok gitlab # [ 8.147154] postgresql-pre-start[837]: selecting dynamic shared memory implementation ... posix gitlab # [ 8.148404] dhcpcd[839]: DUID 00:01:00:01:32:52:95:9a:52:54:00:12:34:56 gitlab # [ 8.149376] dhcpcd[839]: eth0: IAID 00:12:34:56 gitlab # [ 8.150132] dhcpcd[839]: eth0: adding address fe80::5054:ff:fe12:3456 gitlab # [ 8.184697] postgresql-pre-start[837]: selecting default "max_connections" ... 100 gitlab # [ 8.244829] postgresql-pre-start[837]: selecting default "shared_buffers" ... 128MB gitlab # [ 8.545464] systemd[1]: Started gitlab-workhorse.service. gitlab # [ 8.743093] postgresql-pre-start[837]: selecting default time zone ... UTC gitlab # [ 8.745419] postgresql-pre-start[837]: creating configuration files ... ok gitlab # [ 8.848978] gitlab-workhorse[885]: time="2026-10-02T16:17:31Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4 gitlab # [ 8.851893] gitlab-workhorse[885]: time="2026-10-02T16:17:31Z" level=info msg="Using redis/go-redis" gitlab # [ 8.864740] gitlab-workhorse[885]: time="2026-10-02T16:17:31Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix gitlab # [ 8.990982] postgresql-pre-start[837]: running bootstrap script ... ok gitlab # [ 9.512656] dhcpcd[839]: eth0: soliciting a DHCP lease gitlab # [ 9.520704] dhcpcd[839]: eth0: offered 10.0.2.15 from 10.0.2.2 gitlab # [ 9.536417] dhcpcd[839]: eth0: probing address 10.0.2.15/24 gitlab # [ 9.643250] postgresql-pre-start[837]: performing post-bootstrap initialization ... ok gitlab # [ 9.801792] postgresql-pre-start[837]: syncing data to disk ... ok gitlab # [ 9.803382] postgresql-pre-start[837]: initdb: warning: enabling "trust" authentication for local connections gitlab # [ 9.805596] postgresql-pre-start[837]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. gitlab # [ 9.810031] postgresql-pre-start[837]: Success. You can now start the database server using: gitlab # [ 9.811838] postgresql-pre-start[837]: pg_ctl -D /var/lib/postgresql/18 -l logfile start gitlab # [ 10.020702] postgres[911]: [911] LOG: starting PostgreSQL 18.4 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit gitlab # [ 10.023742] postgres[911]: [911] LOG: listening on IPv6 address "::1", port 5432 gitlab # [ 10.026165] postgres[911]: [911] LOG: listening on IPv4 address "127.0.0.1", port 5432 gitlab # [ 10.028050] postgres[911]: [911] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" gitlab # [ 10.031921] postgres[925]: [925] LOG: database system was shut down at 2026-10-02 16:17:32 GMT gitlab # [ 10.042172] postgres[911]: [911] LOG: database system is ready to accept connections gitlab # [ 10.044741] systemd[1]: Started PostgreSQL Server. gitlab # [ 10.049877] systemd[1]: Starting PostgreSQL Setup Scripts... gitlab # [ 10.072846] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[918]: mkdir -p /var/gitlab/state/home/.ssh: OK gitlab # [ 10.075436] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[918]: chmod 700 /var/gitlab/state/home/.ssh: OK gitlab # [ 10.145705] systemd[1]: Finished gitlab-config.service. gitlab # [ 10.149397] systemd[1]: Started gitaly.service. gitlab # [ 10.293743] postgresql-setup-start[949]: CREATE DATABASE gitlab # [ 10.329885] postgresql-setup-start[955]: CREATE ROLE gitlab # [ 10.349304] postgresql-setup-start[957]: ALTER DATABASE gitlab # [ 10.383500] postgresql-setup-start[962]: CREATE ROLE gitlab # [ 10.387502] systemd[1]: Finished PostgreSQL Setup Scripts. gitlab # [ 10.389063] systemd[1]: Reached target PostgreSQL. gitlab # [ 10.392070] systemd[1]: Starting gitlab-db-config.service... gitlab # [ 10.394892] systemd[1]: Starting gitlab-postgresql.service... gitlab # [ 10.561894] gitlab-postgresql-start[977]: CREATE DATABASE gitlab # [ 10.627381] gitlab-postgresql-start[992]: CREATE EXTENSION gitlab # [ 10.726129] gitlab-postgresql-start[995]: CREATE EXTENSION gitlab # [ 10.730650] systemd[1]: Finished gitlab-postgresql.service. gitlab # [ 10.742054] dhcpcd[839]: eth0: soliciting an IPv6 router gitlab # [ 10.743016] dhcpcd[839]: eth0: Router Advertisement from fe80::2 gitlab # [ 10.743936] dhcpcd[839]: eth0: adding address fec0::5054:ff:fe12:3456/64 gitlab # [ 10.745299] dhcpcd[839]: eth0: adding route to fec0::/64 gitlab # [ 10.746089] dhcpcd[839]: eth0: adding default route via fe80::2 gitlab # [ 10.813407] gitaly[948]: time="2026-10-02T16:17:33.662Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/0r0qd84zrxhqq373hk9spv8j9pg0qlnq-gitaly.toml' and fix the errors that are printed." pid=948 gitlab # [ 10.828334] gitaly[948]: time="2026-10-02T16:17:33.663Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=948 gitlab # [ 10.831623] gitaly[948]: time="2026-10-02T16:17:33.663Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=948 gitlab # [ 10.835001] gitaly[948]: time="2026-10-02T16:17:33.663Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=948 gitlab # [ 10.837811] gitaly[948]: time="2026-10-02T16:17:33.663Z" level=info msg="Starting Gitaly" pid=948 version= gitlab # [ 11.057756] gitaly[948]: time="2026-10-02T16:17:33.907Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=243 pid=948 gitlab # [ 11.059682] gitaly[948]: time="2026-10-02T16:17:33.908Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=948 gitlab # [ 11.061454] gitaly[948]: time="2026-10-02T16:17:33.908Z" level=info msg="finished initializing command factory" duration_ms=0 pid=948 gitlab # [ 11.063155] gitaly[948]: time="2026-10-02T16:17:33.908Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-948/git-exec-1361447726.d/git pid=948 gitlab # [ 11.065341] gitaly[948]: time="2026-10-02T16:17:33.915Z" level=info msg="finished detecting git version" duration_ms=6 pid=948 gitlab # [ 11.067029] gitaly[948]: time="2026-10-02T16:17:33.915Z" level=info msg="using Git version" pid=948 version=2.53.g gitlab # [ 11.068630] gitaly[948]: time="2026-10-02T16:17:33.915Z" level=info msg="clearing disk cache object folder" pid=948 storage=default gitlab # [ 11.070204] gitaly[948]: time="2026-10-02T16:17:33.915Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1452725371 pid=948 storage=default gitlab # [ 11.072735] gitaly[948]: time="2026-10-02T16:17:33.915Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1452725371 pid=948 storage=default gitlab # [ 11.075381] gitaly[948]: time="2026-10-02T16:17:33.915Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=948 gitlab # [ 11.078142] gitaly[948]: time="2026-10-02T16:17:33.915Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=948 gitlab # [ 11.081169] gitaly[948]: time="2026-10-02T16:17:33.915Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1452725371 clear_duration_ms=0 pid=948 storage=default gitlab # [ 11.083973] gitaly[948]: time="2026-10-02T16:17:33.929Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=948 gitlab # [ 11.086267] gitaly[948]: time="2026-10-02T16:17:33.930Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=948 schema=unix gitlab # [ 11.088442] gitaly[948]: time="2026-10-02T16:17:33.930Z" level=info msg="listening at address" address=/run/gitaly/gitaly-948/sock.d/intern pid=948 schema=unix gitlab # [ 11.090450] gitaly[948]: time="2026-10-02T16:17:33.930Z" level=info msg="finished tempdir cleaner walk" pid=948 storage=default time_ms=0 gitlab # [ 11.092236] gitaly[948]: time="2026-10-02T16:17:33.930Z" level=info msg="counting repositories" pid=948 gitlab # [ 11.093599] gitaly[948]: time="2026-10-02T16:17:33.930Z" level=info msg="starting to count repositories" pid=948 storage_path=/var/gitlab/state/repositories gitlab # [ 11.095569] gitaly[948]: time="2026-10-02T16:17:33.930Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=948 storage_path=/var/gitlab/state/repositories gitlab # [ 11.097884] gitaly[948]: time="2026-10-02T16:17:33.930Z" level=info msg="completed counting all repositories" counting_duration=0 pid=948 gitlab # [ 11.099676] gitaly[948]: time="2026-10-02T16:17:33.935Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=948 gitlab # [ 11.101459] gitaly[948]: time="2026-10-02T16:17:33.935Z" level=info msg="Started Gitaly" duration_ms=271 pid=948 gitlab # [ 11.102921] gitaly[948]: time="2026-10-02T16:17:33.935Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=948 gitlab # [ 11.104692] gitaly[948]: time="2026-10-02T16:17:33.935Z" level=info msg="maintenance: daily scheduled" pid=948 scheduled="2026-10-03 12:00:00 +0000 UTC" gitlab # [ 11.756396] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[972]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 11.757436] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[972]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 11.776410] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[972]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 14.016366] dhcpcd[839]: eth0: leased 10.0.2.15 for 86400 seconds gitlab # [ 14.016571] dhcpcd[839]: eth0: adding route to 10.0.2.0/24 gitlab # [ 14.016639] dhcpcd[839]: eth0: adding default route via 10.0.2.2 gitlab # [ 14.117066] systemd[1]: Started DHCP Client. gitlab # [ 14.118216] systemd[1]: Reached target Network is Online. gitlab # [ 14.445287] gitaly[948]: time="2026-10-02T16:17:37.294Z" level=info msg="License database preloaded" duration_ms=3631 pid=948 gitlab # [ 58.927012] gitlab-workhorse[885]: time="2026-10-02T16:18:21Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 59.320679] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[972]: W, [2026-10-02T16:18:22.170015 #972] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 62.454298] postgres[1530]: [1530] ERROR: relation "feature_gates" does not exist at character 523 gitlab # [ 62.454535] postgres[1530]: [1530] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), gitlab # [ 62.454600] postgres[1530]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, gitlab # [ 62.454660] postgres[1530]: c.collname, col_description(a.attrelid, a.attnum) AS comment, gitlab # [ 62.454718] postgres[1530]: attidentity AS identity, gitlab # [ 62.454776] postgres[1530]: attgenerated as attgenerated gitlab # [ 62.454852] postgres[1530]: FROM pg_attribute a gitlab # [ 62.454912] postgres[1530]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum gitlab # [ 62.454970] postgres[1530]: LEFT JOIN pg_type t ON a.atttypid = t.oid gitlab # [ 62.455028] postgres[1530]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation gitlab # [ 62.455086] postgres[1530]: WHERE a.attrelid = '"feature_gates"'::regclass gitlab # [ 62.455144] postgres[1530]: AND a.attnum > 0 AND NOT a.attisdropped gitlab # [ 62.455200] postgres[1530]: ORDER BY a.attnum gitlab # [ 62.455256] postgres[1530]: gitlab # [ 68.212582] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1581]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping gitlab # [ 68.212825] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1581]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping gitlab # [ 68.937489] gitlab-workhorse[885]: time="2026-10-02T16:18:31Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 78.938173] gitlab-workhorse[885]: time="2026-10-02T16:18:41Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 88.948073] gitlab-workhorse[885]: time="2026-10-02T16:18:51Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 90.914243] postgres[1684]: [1684] ERROR: relation "table_without_models" does not exist gitlab # [ 90.914567] postgres[1684]: [1684] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 98.957888] gitlab-workhorse[885]: time="2026-10-02T16:19:01Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 108.966190] gitlab-workhorse[885]: time="2026-10-02T16:19:11Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 109.025096] systemd[1]: Finished gitlab-db-config.service. gitlab # [ 109.028861] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 109.032150] systemd[1]: Starting gitlab.service... gitlab # [ 109.615561] bundle[1941]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 109.615891] bundle[1941]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 109.617793] bundle[1941]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 114.543197] sidekiq-cluster[1940]: {"severity":"INFO","time":"2026-10-02T16:19:17.361Z","message":"Starting cluster with 1 processes"} gitlab # [ 114.648923] bundle[1941]: {"timestamp":"2026-10-02T16:19:17.467Z","pid":1941,"message":"* Enabling systemd notification integration"} gitlab # [ 114.651649] bundle[1941]: {"timestamp":"2026-10-02T16:19:17.498Z","pid":1941,"message":"Puma starting in cluster mode..."} gitlab # [ 114.654056] bundle[1941]: {"timestamp":"2026-10-02T16:19:17.498Z","pid":1941,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"} gitlab # [ 114.656563] bundle[1941]: {"timestamp":"2026-10-02T16:19:17.498Z","pid":1941,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} gitlab # [ 114.659578] bundle[1941]: {"timestamp":"2026-10-02T16:19:17.498Z","pid":1941,"message":"* Min threads: 0"} gitlab # [ 114.661674] bundle[1941]: {"timestamp":"2026-10-02T16:19:17.498Z","pid":1941,"message":"* Max threads: 4"} gitlab # [ 114.663742] bundle[1941]: {"timestamp":"2026-10-02T16:19:17.498Z","pid":1941,"message":"* Environment: production"} gitlab # [ 114.666069] bundle[1941]: {"timestamp":"2026-10-02T16:19:17.498Z","pid":1941,"message":"* Master PID: 1941"} gitlab # [ 114.668281] bundle[1941]: {"timestamp":"2026-10-02T16:19:17.498Z","pid":1941,"message":"* Workers: 2"} gitlab # [ 114.670311] bundle[1941]: {"timestamp":"2026-10-02T16:19:17.498Z","pid":1941,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} gitlab # [ 114.672966] bundle[1941]: {"timestamp":"2026-10-02T16:19:17.498Z","pid":1941,"message":"* Preloading application"} gitlab # [ 118.966523] gitlab-workhorse[885]: time="2026-10-02T16:19:21Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 128.973540] gitlab-workhorse[885]: time="2026-10-02T16:19:31Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 138.974992] gitlab-workhorse[885]: time="2026-10-02T16:19:41Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 148.975891] gitlab-workhorse[885]: time="2026-10-02T16:19:51Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 155.467365] bundle[1941]: W, [2026-10-02T16:19:58.317094 #1941] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 158.985754] gitlab-workhorse[885]: time="2026-10-02T16:20:01Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 159.721627] sidekiq-cluster[1998]: 2026-10-02T16:20:02.571Z pid=1998 tid=wo6 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1998, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>426950656} gitlab # [ 160.078705] sidekiq-cluster[1998]: W, [2026-10-02T16:20:02.928783 #1998] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 160.151610] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:03.001Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 163.793884] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.643Z","message":"GitLab reliable fetch activated!"} gitlab # [ 163.794129] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.643Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 163.826283] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.676Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} gitlab # [ 163.830109] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.679Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"} gitlab # [ 163.833864] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.682Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} gitlab # [ 163.837338] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.684Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} gitlab # [ 163.841173] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.686Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} gitlab # [ 163.844667] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.689Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} gitlab # [ 163.848211] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.691Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} gitlab # [ 163.851414] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.693Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} gitlab # [ 163.854890] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.694Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} gitlab # [ 163.858310] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.696Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} gitlab # [ 163.862090] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.698Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} gitlab # [ 163.865680] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.701Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} gitlab # [ 163.868996] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.703Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} gitlab # [ 163.872546] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.705Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} gitlab # [ 163.875944] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.707Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} gitlab # [ 163.879484] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.709Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} gitlab # [ 163.882663] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.711Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} gitlab # [ 163.885572] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.714Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} gitlab # [ 163.888631] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.717Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} gitlab # [ 163.891422] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.719Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} gitlab # [ 163.894600] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.722Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} gitlab # [ 163.897743] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.724Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} gitlab # [ 163.900345] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.726Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} gitlab # [ 163.902924] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.729Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} gitlab # [ 163.905604] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.731Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} gitlab # [ 163.907997] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.733Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} gitlab # [ 163.910339] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.736Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} gitlab # [ 163.912741] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.739Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} gitlab # [ 163.915115] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.741Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} gitlab # [ 163.917678] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.744Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} gitlab # [ 163.919872] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.746Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} gitlab # [ 163.922488] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.749Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} gitlab # [ 163.924632] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.751Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} gitlab # [ 163.926883] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.754Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} gitlab # [ 163.929165] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.755Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} gitlab # [ 163.931364] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.757Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} gitlab # [ 163.933695] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.760Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} gitlab # [ 163.935890] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.762Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} gitlab # [ 163.938157] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.764Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} gitlab # [ 163.940233] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.766Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} gitlab # [ 163.942442] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.768Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} gitlab # [ 163.944655] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.770Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} gitlab # [ 163.946856] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.773Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} gitlab # [ 163.949078] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.775Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} gitlab # [ 163.951350] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.777Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} gitlab # [ 163.953804] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.780Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} gitlab # [ 163.956060] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.781Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} gitlab # [ 163.958274] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.784Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} gitlab # [ 163.960556] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.786Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} gitlab # [ 163.962773] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.788Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} gitlab # [ 163.965053] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.790Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} gitlab # [ 163.967233] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.793Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} gitlab # [ 163.969512] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.805Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} gitlab # [ 163.971957] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.808Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} gitlab # [ 163.974379] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.810Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} gitlab # [ 163.976671] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.813Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} gitlab # [ 163.978907] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.815Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} gitlab # [ 163.981402] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.818Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} gitlab # [ 163.983721] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.820Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} gitlab # [ 163.986200] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.822Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} gitlab # [ 164.050343] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.900Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} gitlab # [ 164.053795] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.902Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} gitlab # [ 164.057304] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.905Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} gitlab # [ 164.060707] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.909Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} gitlab # [ 164.064267] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.911Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} gitlab # [ 164.067594] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.915Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} gitlab # [ 164.070837] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.917Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} gitlab # [ 164.074021] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.920Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} gitlab # [ 164.077204] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.921Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} gitlab # [ 164.080258] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.923Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} gitlab # [ 164.083179] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.926Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} gitlab # [ 164.086106] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.928Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} gitlab # [ 164.088874] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.930Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} gitlab # [ 164.091640] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.932Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} gitlab # [ 164.094513] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.935Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} gitlab # [ 164.097200] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.937Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} gitlab # [ 164.099679] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.939Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} gitlab # [ 164.102619] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.941Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} gitlab # [ 164.105052] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.943Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} gitlab # [ 164.107363] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.946Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} gitlab # [ 164.109985] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.947Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} gitlab # [ 164.112574] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.950Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} gitlab # [ 164.114943] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.952Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} gitlab # [ 164.117316] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.955Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} gitlab # [ 164.119745] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.957Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} gitlab # [ 164.122291] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.960Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} gitlab # [ 164.124787] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.962Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} gitlab # [ 164.127139] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.964Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} gitlab # [ 164.129633] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.966Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} gitlab # [ 164.131991] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.968Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} gitlab # [ 164.134554] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.970Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} gitlab # [ 164.136947] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.972Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} gitlab # [ 164.139313] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.974Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} gitlab # [ 164.141961] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.976Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} gitlab # [ 164.144058] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.979Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} gitlab # [ 164.146487] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.982Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} gitlab # [ 164.148928] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.984Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} gitlab # [ 164.151222] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:06.986Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} gitlab # [ 164.153606] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:07.000Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} gitlab # [ 164.155863] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:07.002Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} gitlab # [ 164.158267] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:07.004Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} gitlab # [ 164.160657] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:07.008Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} gitlab # [ 164.162822] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:07.012Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} gitlab # [ 164.165204] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:07.014Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} gitlab # [ 164.167982] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:07.018Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} gitlab # [ 164.170622] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:07.019Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} gitlab # [ 164.172899] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:07.021Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} gitlab # [ 164.175248] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:07.024Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} gitlab # [ 164.177657] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:07.027Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} gitlab # [ 168.993418] gitlab-workhorse[885]: time="2026-10-02T16:20:11Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 178.994190] gitlab-workhorse[885]: time="2026-10-02T16:20:21Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 188.995559] gitlab-workhorse[885]: time="2026-10-02T16:20:31Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 198.996363] gitlab-workhorse[885]: time="2026-10-02T16:20:41Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 200.679124] postgres[2764]: [2764] ERROR: relation "table_without_models" does not exist gitlab # [ 200.679364] postgres[2764]: [2764] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 205.548300] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:48.397Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 205.551362] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:48.401Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} gitlab # [ 205.554820] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:48.401Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 205.557723] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:48.401Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 205.560982] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:48.403Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 207.504897] postgres[2821]: [2821] ERROR: relation "table_without_models" does not exist gitlab # [ 207.505119] postgres[2821]: [2821] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 208.253308] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:51.102Z","message":"Cleaning working queues"} gitlab # [ 208.255828] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:51.105Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"7dbc56eac6f58e77854666ab","created_at":"2026-10-02T16:19:11.416Z","trace_propagation_headers":{"sentry-trace":"2c2a0c8162824a79a8a9e3049500a12e-bac14e13d2664616","baggage":"sentry-trace_id=2c2a0c8162824a79a8a9e3049500a12e,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"02a5b8ca6bf42e159e45933086377a24","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:11.428Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-7dbc56eac6f58e77854666ab: start","job_status":"start","queue_duration_s":99.677096,"scheduling_latency_s":99.67715} gitlab # [ 208.449967] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:20:51.299Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"7dbc56eac6f58e77854666ab","created_at":"2026-10-02T16:19:11.416Z","trace_propagation_headers":{"sentry-trace":"2c2a0c8162824a79a8a9e3049500a12e-bac14e13d2664616","baggage":"sentry-trace_id=2c2a0c8162824a79a8a9e3049500a12e,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"02a5b8ca6bf42e159e45933086377a24","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-02T16:19:11.428Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-7dbc56eac6f58e77854666ab: done: 0.194145 sec","job_status":"done","queue_duration_s":99.677096,"scheduling_latency_s":99.67715,"redis_calls":29,"redis_duration_s":0.006255999999999999,"redis_read_bytes":282,"redis_write_bytes":3632,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.003469,"redis_feature_flag_read_bytes":272,"redis_feature_flag_write_bytes":2385,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001536,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001251,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":15,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.094868,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.10950696,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.194145,"completed_at":"2026-10-02T16:20:51.299Z","load_balancing_strategy":"primary","db_duration_s":0.061068,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 209.006014] gitlab-workhorse[885]: time="2026-10-02T16:20:51Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 214.115296] bundle[1941]: {"timestamp":"2026-10-02T16:20:56.964Z","pid":1941,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} gitlab # [ 214.115531] bundle[1941]: {"timestamp":"2026-10-02T16:20:56.965Z","pid":1941,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} gitlab # [ 214.115638] bundle[1941]: {"timestamp":"2026-10-02T16:20:56.965Z","pid":1941,"message":"! #\u003cThread:0x0000e447a2937c08 /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} gitlab # [ 214.115715] bundle[1941]: {"timestamp":"2026-10-02T16:20:56.965Z","pid":1941,"message":"! #\u003cThread:0x0000e447a2b78f08 /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} gitlab # [ 214.115775] bundle[1941]: {"timestamp":"2026-10-02T16:20:56.965Z","pid":1941,"message":"Use Ctrl-C to stop"} gitlab # [ 214.115833] bundle[1941]: {"timestamp":"2026-10-02T16:20:56.965Z","pid":1941,"message":"Sending status to systemd every 1.0 sec"} gitlab # [ 214.674763] bundle[1941]: {"timestamp":"2026-10-02T16:20:57.524Z","pid":1941,"message":"- Worker 0 (PID: 2921) booted in 0.52s, phase: 0"} gitlab # [ 214.703718] bundle[1941]: {"timestamp":"2026-10-02T16:20:57.553Z","pid":1941,"message":"- Worker 1 (PID: 2925) booted in 0.53s, phase: 0"} gitlab # [ 214.707849] systemd[1]: Started gitlab.service. gitlab # [ 214.708985] systemd[1]: Reached target Common target for all GitLab services.. gitlab # [ 214.712153] systemd[1]: Starting setup-gitlab.service... gitlab # [ 214.745796] setup-gitlab-start[2958]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' gitlab # [ 218.632731] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:21:01 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 3856 gitlab # [ 218.636628] setup-gitlab-start[2958]: + TOKEN=nixbot-test-token-of-20-chars gitlab # [ 218.637685] setup-gitlab-start[2958]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n' gitlab # [ 218.648665] runuser[2984]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) gitlab # [ 219.204829] setup-gitlab-start[2985]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 219.208334] setup-gitlab-start[2985]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 219.211620] setup-gitlab-start[2985]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 220.060509] setup-gitlab-start[2985]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 220.060910] setup-gitlab-start[2985]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 220.067520] setup-gitlab-start[2985]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 220.365636] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:03.214Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"f2bd95e9dca3143885288424","created_at":"2026-10-02T16:21:03.144Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"40ae272f4e75b179facda16dffa6b057","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.211Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-f2bd95e9dca3143885288424: start","job_status":"start","queue_duration_s":0.002795,"scheduling_latency_s":0.002836} gitlab # [ 220.456483] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:03.306Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 220.490380] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:03.340Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"f2bd95e9dca3143885288424","created_at":"2026-10-02T16:21:03.144Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"40ae272f4e75b179facda16dffa6b057","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.211Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-f2bd95e9dca3143885288424: done: 0.125354 sec","job_status":"done","queue_duration_s":0.002795,"scheduling_latency_s":0.002836,"redis_calls":10,"redis_duration_s":0.024567,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001404,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.02097,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002193,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023051,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01315,"duration_s":0.125354,"completed_at":"2026-10-02T16:21:03.339Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024638,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 220.551571] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:03.344Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"9ee779919b5fdd1b1ccdb4a2","created_at":"2026-10-02T16:21:03.222Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bfed2ed856d563aa9432a18ca90e5019","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.236Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-9ee779919b5fdd1b1ccdb4a2: start","job_status":"start","queue_duration_s":0.108756,"scheduling_latency_s":0.108785} gitlab # [ 220.569677] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:03.407Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"9ee779919b5fdd1b1ccdb4a2","created_at":"2026-10-02T16:21:03.222Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bfed2ed856d563aa9432a18ca90e5019","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.236Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-9ee779919b5fdd1b1ccdb4a2: done: 0.06303 sec","job_status":"done","queue_duration_s":0.108756,"scheduling_latency_s":0.108785,"redis_calls":13,"redis_duration_s":0.003981,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001267,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001595,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001119,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03347,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0169069,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.06303,"completed_at":"2026-10-02T16:21:03.407Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012506,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 220.618451] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:03.430Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"afcadcf71541acd983d48447","created_at":"2026-10-02T16:21:03.250Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c7710de422b5e11e10473d1a324bb0b6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.256Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-afcadcf71541acd983d48447: start","job_status":"start","queue_duration_s":0.174222,"scheduling_latency_s":0.174253} gitlab # [ 220.633456] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:03.479Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 220.641347] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:03.490Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"afcadcf71541acd983d48447","created_at":"2026-10-02T16:21:03.250Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c7710de422b5e11e10473d1a324bb0b6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.256Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-afcadcf71541acd983d48447: done: 0.06014 sec","job_status":"done","queue_duration_s":0.174222,"scheduling_latency_s":0.174253,"redis_calls":10,"redis_duration_s":0.001924,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000371,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001158,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000395,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015517,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029168,"duration_s":0.06014,"completed_at":"2026-10-02T16:21:03.490Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027016,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 220.684868] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:03.494Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"839f7a4d765f6a8fc4da217f","created_at":"2026-10-02T16:21:03.262Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"73ed4c1fecd1c4acdce1462caefca897","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.267Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-839f7a4d765f6a8fc4da217f: start","job_status":"start","queue_duration_s":0.226209,"scheduling_latency_s":0.226237} gitlab # [ 220.716245] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:03.566Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"839f7a4d765f6a8fc4da217f","created_at":"2026-10-02T16:21:03.262Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"73ed4c1fecd1c4acdce1462caefca897","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.267Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-839f7a4d765f6a8fc4da217f: done: 0.071912 sec","job_status":"done","queue_duration_s":0.226209,"scheduling_latency_s":0.226237,"redis_calls":10,"redis_duration_s":0.006616,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000554,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004398,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001664,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021835,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00192976,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.071912,"completed_at":"2026-10-02T16:21:03.565Z","load_balancing_strategy":"primary","db_duration_s":0.031873,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 220.775873] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:03.585Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"38a419018d6805ac713abef4","created_at":"2026-10-02T16:21:03.271Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ce55f3e78ca037cc01d79739a1e981c6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.279Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-38a419018d6805ac713abef4: start","job_status":"start","queue_duration_s":0.305662,"scheduling_latency_s":0.305692} gitlab # [ 220.829764] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:03.679Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 220.859532] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:03.709Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"38a419018d6805ac713abef4","created_at":"2026-10-02T16:21:03.271Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ce55f3e78ca037cc01d79739a1e981c6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.279Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-38a419018d6805ac713abef4: done: 0.124167 sec","job_status":"done","queue_duration_s":0.305662,"scheduling_latency_s":0.305692,"redis_calls":10,"redis_duration_s":0.011783,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001381,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006157,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004245,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018517,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00053872,"duration_s":0.124167,"completed_at":"2026-10-02T16:21:03.709Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003699,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 220.917602] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:03.715Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"6b38503f89218a17717d7ea3","created_at":"2026-10-02T16:21:03.282Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"967998f19e62acf24dae8c6d81ce7324","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.289Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-6b38503f89218a17717d7ea3: start","job_status":"start","queue_duration_s":0.426202,"scheduling_latency_s":0.42624} gitlab # [ 220.935295] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:03.784Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"6b38503f89218a17717d7ea3","created_at":"2026-10-02T16:21:03.282Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"967998f19e62acf24dae8c6d81ce7324","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.289Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-6b38503f89218a17717d7ea3: done: 0.069425 sec","job_status":"done","queue_duration_s":0.426202,"scheduling_latency_s":0.42624,"redis_calls":12,"redis_duration_s":0.005191,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.003098,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001569,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000524,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024772,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00038756,"duration_s":0.069425,"completed_at":"2026-10-02T16:21:03.784Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007498,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 220.980073] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:03.787Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"d4dd195948b084d8775e34ae","created_at":"2026-10-02T16:21:03.293Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f09a058d7512909618dd6d23f9568c02","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.301Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-d4dd195948b084d8775e34ae: start","job_status":"start","queue_duration_s":0.485181,"scheduling_latency_s":0.485221} gitlab # [ 221.067432] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:03.916Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"d4dd195948b084d8775e34ae","created_at":"2026-10-02T16:21:03.293Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f09a058d7512909618dd6d23f9568c02","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.301Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-d4dd195948b084d8775e34ae: done: 0.129847 sec","job_status":"done","queue_duration_s":0.485181,"scheduling_latency_s":0.485221,"redis_calls":14,"redis_duration_s":0.005118,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002811,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001216,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001091,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.078989,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0008844,"exclusive_lock_hold_duration_s":0.08442485999999999,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.129847,"completed_at":"2026-10-02T16:21:03.916Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020755,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 221.130993] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:03.919Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"df148e6430058690cef3b1fd","created_at":"2026-10-02T16:21:03.329Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba4d6b76562448e56112e1ad2c2875ab","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.343Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-df148e6430058690cef3b1fd: start","job_status":"start","queue_duration_s":0.575981,"scheduling_latency_s":0.576005} gitlab # [ 221.146324] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:03.970Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"df148e6430058690cef3b1fd","created_at":"2026-10-02T16:21:03.329Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba4d6b76562448e56112e1ad2c2875ab","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.343Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-df148e6430058690cef3b1fd: done: 0.050613 sec","job_status":"done","queue_duration_s":0.575981,"scheduling_latency_s":0.576005,"redis_calls":16,"redis_duration_s":0.002899,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001515,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001003,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000381,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029022,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000334,"duration_s":0.050613,"completed_at":"2026-10-02T16:21:03.970Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012631,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 221.190684] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:03.972Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"c7022c29610d2a2783234abe","created_at":"2026-10-02T16:21:03.369Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b757be5f60897438ecb79e18bedcd17d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.404Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-c7022c29610d2a2783234abe: start","job_status":"start","queue_duration_s":0.567547,"scheduling_latency_s":0.567568} gitlab # [ 221.204585] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:04.004Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"c7022c29610d2a2783234abe","created_at":"2026-10-02T16:21:03.369Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b757be5f60897438ecb79e18bedcd17d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.404Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-c7022c29610d2a2783234abe: done: 0.03192 sec","job_status":"done","queue_duration_s":0.567547,"scheduling_latency_s":0.567568,"redis_calls":12,"redis_duration_s":0.002074,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000325,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000855,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000894,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01945,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00067612,"exclusive_lock_hold_duration_s":0.00883214,"duration_s":0.03192,"completed_at":"2026-10-02T16:21:04.004Z","load_balancing_strategy":"primary","db_duration_s":0.004282,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 221.247400] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:04.005Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"6161a3516ab0882baacce49c","created_at":"2026-10-02T16:21:03.428Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9c4961f43b82b6624ff74f46c4e74fd7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T16:21:03.451Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-6161a3516ab0882baacce49c: start","job_status":"start","queue_duration_s":0.554168,"scheduling_latency_s":0.554187} gitlab # [ 221.261465] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:04.017Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 221.264871] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:04.024Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"6161a3516ab0882baacce49c","created_at":"2026-10-02T16:21:03.428Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9c4961f43b82b6624ff74f46c4e74fd7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T16:21:03.451Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-6161a3516ab0882baacce49c: done: 0.019216 sec","job_status":"done","queue_duration_s":0.554168,"scheduling_latency_s":0.554187,"redis_calls":10,"redis_duration_s":0.001372,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000272,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000794,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000306,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011384,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024184,"duration_s":0.019216,"completed_at":"2026-10-02T16:21:04.024Z","load_balancing_strategy":"primary","db_duration_s":0.000427,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 221.307465] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:04.026Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"b2d0d58c0ed8ae7e468a62b9","created_at":"2026-10-02T16:21:03.475Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bf8e0710628f5efa6efcc14207049e4f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.498Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-b2d0d58c0ed8ae7e468a62b9: start","job_status":"start","queue_duration_s":0.527796,"scheduling_latency_s":0.527813} gitlab # [ 221.322785] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:04.045Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"b2d0d58c0ed8ae7e468a62b9","created_at":"2026-10-02T16:21:03.475Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bf8e0710628f5efa6efcc14207049e4f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.498Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-b2d0d58c0ed8ae7e468a62b9: done: 0.019032 sec","job_status":"done","queue_duration_s":0.527796,"scheduling_latency_s":0.527813,"redis_calls":10,"redis_duration_s":0.001518,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000286,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000933,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000299,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011528,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026242,"duration_s":0.019032,"completed_at":"2026-10-02T16:21:04.045Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000401,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 221.366917] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:04.046Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"7911be3b05ebd837b09d11ea","created_at":"2026-10-02T16:21:03.529Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4bd5688216d5c900911ef19f9f6cfd6c","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.540Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-7911be3b05ebd837b09d11ea: start","job_status":"start","queue_duration_s":0.505481,"scheduling_latency_s":0.5055} gitlab # [ 221.381522] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:04.070Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"7911be3b05ebd837b09d11ea","created_at":"2026-10-02T16:21:03.529Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4bd5688216d5c900911ef19f9f6cfd6c","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.540Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-7911be3b05ebd837b09d11ea: done: 0.024434 sec","job_status":"done","queue_duration_s":0.505481,"scheduling_latency_s":0.5055,"redis_calls":10,"redis_duration_s":0.001349,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000256,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000809,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000284,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014846,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00023446,"duration_s":0.024434,"completed_at":"2026-10-02T16:21:04.070Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002778,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 221.425383] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:04.072Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"b76dcd9ac290ac14c4f7124d","created_at":"2026-10-02T16:21:03.548Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e3223d257e9951ccfd02dd41e33f83c7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T16:21:03.558Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-b76dcd9ac290ac14c4f7124d: start","job_status":"start","queue_duration_s":0.513565,"scheduling_latency_s":0.513583} gitlab # [ 221.439029] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:04.103Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"b76dcd9ac290ac14c4f7124d","created_at":"2026-10-02T16:21:03.548Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e3223d257e9951ccfd02dd41e33f83c7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T16:21:03.558Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-b76dcd9ac290ac14c4f7124d: done: 0.031271 sec","job_status":"done","queue_duration_s":0.513565,"scheduling_latency_s":0.513583,"redis_calls":10,"redis_duration_s":0.00135,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000282,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000782,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000286,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02023,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026762,"duration_s":0.031271,"completed_at":"2026-10-02T16:21:04.103Z","load_balancing_strategy":"primary","db_duration_s":0.004089,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 221.481106] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:04.104Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"e527af3026659eb3d47d32ac","created_at":"2026-10-02T16:21:03.580Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"efc3675b350adc4334d0ba69f7549482","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.587Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-e527af3026659eb3d47d32ac: start","job_status":"start","queue_duration_s":0.516589,"scheduling_latency_s":0.516606} gitlab # [ 221.511363] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:04.361Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"e527af3026659eb3d47d32ac","created_at":"2026-10-02T16:21:03.580Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"efc3675b350adc4334d0ba69f7549482","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.587Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-e527af3026659eb3d47d32ac: done: 0.256658 sec","job_status":"done","queue_duration_s":0.516589,"scheduling_latency_s":0.516606,"redis_calls":12,"redis_duration_s":0.002015,"redis_read_bytes":11,"redis_write_bytes":1370,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000283,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000927,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000805,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.035,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.035,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.210579,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00051596,"exclusive_lock_hold_duration_s":0.23648592000000002,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.256658,"completed_at":"2026-10-02T16:21:04.361Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.052463,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 221.557248] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:04.362Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"0515639d75f7989633264037","created_at":"2026-10-02T16:21:03.619Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"45389b482dddca05ad4fb509662ec1d6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T16:21:03.625Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-0515639d75f7989633264037: start","job_status":"start","queue_duration_s":0.736773,"scheduling_latency_s":0.736791} gitlab # [ 221.571122] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:04.374Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 221.575014] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:04.382Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"0515639d75f7989633264037","created_at":"2026-10-02T16:21:03.619Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"45389b482dddca05ad4fb509662ec1d6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T16:21:03.625Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-0515639d75f7989633264037: done: 0.019467 sec","job_status":"done","queue_duration_s":0.736773,"scheduling_latency_s":0.736791,"redis_calls":10,"redis_duration_s":0.001671,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000345,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000976,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00035,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026358,"duration_s":0.019467,"completed_at":"2026-10-02T16:21:04.381Z","load_balancing_strategy":"primary","db_duration_s":0.000725,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 221.618634] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:04.383Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"b9dea41c5162e1ce05e2319e","created_at":"2026-10-02T16:21:03.638Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"679fdfe9c0db0b872cf690164cea6000","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.644Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-b9dea41c5162e1ce05e2319e: start","job_status":"start","queue_duration_s":0.738708,"scheduling_latency_s":0.738725} gitlab # [ 221.844226] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:04.693Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"b9dea41c5162e1ce05e2319e","created_at":"2026-10-02T16:21:03.638Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"679fdfe9c0db0b872cf690164cea6000","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.644Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-b9dea41c5162e1ce05e2319e: done: 0.310559 sec","job_status":"done","queue_duration_s":0.738708,"scheduling_latency_s":0.738725,"redis_calls":565,"redis_duration_s":0.059418,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000345,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000884,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.058189,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.283339,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025428,"duration_s":0.310559,"completed_at":"2026-10-02T16:21:04.693Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000495,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 221.910207] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:04.695Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"794daaaab7342012d759ace3","created_at":"2026-10-02T16:21:03.652Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eb4295d209f4d178eee28d8c91a942a3","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.665Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-794daaaab7342012d759ace3: start","job_status":"start","queue_duration_s":1.029928,"scheduling_latency_s":1.029946} gitlab # [ 221.928391] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:04.717Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"794daaaab7342012d759ace3","created_at":"2026-10-02T16:21:03.652Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eb4295d209f4d178eee28d8c91a942a3","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.665Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-794daaaab7342012d759ace3: done: 0.022494 sec","job_status":"done","queue_duration_s":1.029928,"scheduling_latency_s":1.029946,"redis_calls":12,"redis_duration_s":0.002025,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000342,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000752,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013145,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00048128,"exclusive_lock_hold_duration_s":0.00336988,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.022494,"completed_at":"2026-10-02T16:21:04.717Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001825,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 221.973999] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:04.719Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"6f7caeab41b3562aed60cf02","created_at":"2026-10-02T16:21:03.678Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.696Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-6f7caeab41b3562aed60cf02: start","job_status":"start","queue_duration_s":1.022489,"scheduling_latency_s":1.022507} gitlab # [ 221.988043] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:04.825Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"6f7caeab41b3562aed60cf02","created_at":"2026-10-02T16:21:03.678Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.696Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-6f7caeab41b3562aed60cf02: done: 0.105764 sec","job_status":"done","queue_duration_s":1.022489,"scheduling_latency_s":1.022507,"redis_calls":71,"redis_duration_s":0.012534,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.00168,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000766,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.009553,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000535,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.092095,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027092,"duration_s":0.105764,"completed_at":"2026-10-02T16:21:04.824Z","load_balancing_strategy":"primary","db_duration_s":0.00206,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 222.032699] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:04.826Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"c7adb7134d07f21ec63b9c07","created_at":"2026-10-02T16:21:03.711Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"31ad40142d8b0ee9c34232a44c8a375d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.721Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-c7adb7134d07f21ec63b9c07: start","job_status":"start","queue_duration_s":1.105297,"scheduling_latency_s":1.105314} gitlab # [ 222.274257] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.124Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"c7adb7134d07f21ec63b9c07","created_at":"2026-10-02T16:21:03.711Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"31ad40142d8b0ee9c34232a44c8a375d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.721Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-c7adb7134d07f21ec63b9c07: done: 0.297682 sec","job_status":"done","queue_duration_s":1.105297,"scheduling_latency_s":1.105314,"redis_calls":12,"redis_duration_s":0.004834,"redis_read_bytes":11,"redis_write_bytes":1447,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000371,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":319,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002916,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001547,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":542,"db_count":101,"db_write_count":0,"db_cached_count":99,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":101,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":99,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":100,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":99,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.272071,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00053684,"exclusive_lock_hold_duration_s":0.27386472,"extra.ci_update_locked_unknown_artifacts_worker.removed_count":0,"extra.ci_update_locked_unknown_artifacts_worker.locked_count":0,"duration_s":0.297682,"completed_at":"2026-10-02T16:21:05.124Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016002,"urgency":"throttled","target_duration_s":300,"target_scheduling_latency_s":null} gitlab # [ 222.342777] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.125Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"6c085efe1250c2662389cd35","created_at":"2026-10-02T16:21:03.725Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"74db2be9de1f7263301e7b2798c88660","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.734Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-6c085efe1250c2662389cd35: start","job_status":"start","queue_duration_s":1.391388,"scheduling_latency_s":1.391406} gitlab # [ 222.359466] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.164Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"6c085efe1250c2662389cd35","created_at":"2026-10-02T16:21:03.725Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"74db2be9de1f7263301e7b2798c88660","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.734Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-6c085efe1250c2662389cd35: done: 0.038542 sec","job_status":"done","queue_duration_s":1.391388,"scheduling_latency_s":1.391406,"redis_calls":19,"redis_duration_s":0.003545,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000802,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000269,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.001836,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000638,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027314,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00030974,"duration_s":0.038542,"completed_at":"2026-10-02T16:21:05.163Z","load_balancing_strategy":"primary","db_duration_s":0.002727,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 222.404933] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.165Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"977a47a82b936037eeda8f94","created_at":"2026-10-02T16:21:03.759Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f7bf220c81a8287142bf8b978237cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.781Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-977a47a82b936037eeda8f94: start","job_status":"start","queue_duration_s":1.383976,"scheduling_latency_s":1.383993} gitlab # [ 222.420988] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.211Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"977a47a82b936037eeda8f94","created_at":"2026-10-02T16:21:03.759Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f7bf220c81a8287142bf8b978237cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.781Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-977a47a82b936037eeda8f94: done: 0.04577 sec","job_status":"done","queue_duration_s":1.383976,"scheduling_latency_s":1.383993,"redis_calls":12,"redis_duration_s":0.003404,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000386,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001903,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001115,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020868,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00074572,"exclusive_lock_hold_duration_s":0.01710536,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.04577,"completed_at":"2026-10-02T16:21:05.211Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016271,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 222.466810] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.212Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"5cc60b6dab21c39ed8a1a6b0","created_at":"2026-10-02T16:21:03.810Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d06b64c85f12b533d0dd19f4c964040b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.824Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-5cc60b6dab21c39ed8a1a6b0: start","job_status":"start","queue_duration_s":1.388461,"scheduling_latency_s":1.388479} gitlab # [ 222.482053] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.224Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 222.485559] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.231Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"5cc60b6dab21c39ed8a1a6b0","created_at":"2026-10-02T16:21:03.810Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d06b64c85f12b533d0dd19f4c964040b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T16:21:03.824Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-5cc60b6dab21c39ed8a1a6b0: done: 0.018805 sec","job_status":"done","queue_duration_s":1.388461,"scheduling_latency_s":1.388479,"redis_calls":10,"redis_duration_s":0.001805,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000411,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001024,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00037,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010591,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002987,"duration_s":0.018805,"completed_at":"2026-10-02T16:21:05.231Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000532,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 222.529177] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.233Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d421719781427561ca401f7e","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d421719781427561ca401f7e: start","job_status":"start","queue_duration_s":0.416587,"scheduling_latency_s":0.416603} gitlab # [ 222.542030] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.256Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d421719781427561ca401f7e","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d421719781427561ca401f7e: done: 0.023124 sec","job_status":"done","queue_duration_s":0.416587,"scheduling_latency_s":0.416603,"redis_calls":16,"redis_duration_s":0.00287,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000397,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000877,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001596,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01424,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026376,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023124,"completed_at":"2026-10-02T16:21:05.256Z","load_balancing_strategy":"primary","db_duration_s":0.000488,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 222.583909] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.257Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"49a3284b5ef804d0849b46c9","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-49a3284b5ef804d0849b46c9: start","job_status":"start","queue_duration_s":0.440956,"scheduling_latency_s":0.440972} gitlab # [ 222.596841] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.272Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"49a3284b5ef804d0849b46c9","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-49a3284b5ef804d0849b46c9: done: 0.014707 sec","job_status":"done","queue_duration_s":0.440956,"scheduling_latency_s":0.440972,"redis_calls":13,"redis_duration_s":0.002272,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000661,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001611,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009311,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014707,"completed_at":"2026-10-02T16:21:05.272Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 222.637969] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.273Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5b7c8a50d4da8d7e0858007","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5b7c8a50d4da8d7e0858007: start","job_status":"start","queue_duration_s":0.456898,"scheduling_latency_s":0.456914} gitlab # [ 222.650858] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.288Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5b7c8a50d4da8d7e0858007","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5b7c8a50d4da8d7e0858007: done: 0.014838 sec","job_status":"done","queue_duration_s":0.456898,"scheduling_latency_s":0.456914,"redis_calls":13,"redis_duration_s":0.0020759999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000725,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001351,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009102,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014838,"completed_at":"2026-10-02T16:21:05.288Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 222.691506] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.290Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8ba30cbe400e9fb93dc401f","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8ba30cbe400e9fb93dc401f: start","job_status":"start","queue_duration_s":0.473656,"scheduling_latency_s":0.473672} gitlab # [ 222.704440] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.304Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8ba30cbe400e9fb93dc401f","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8ba30cbe400e9fb93dc401f: done: 0.014492 sec","job_status":"done","queue_duration_s":0.473656,"scheduling_latency_s":0.473672,"redis_calls":13,"redis_duration_s":0.00192,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000673,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001247,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009648,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014492,"completed_at":"2026-10-02T16:21:05.304Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 222.744924] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.305Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2fed0338d4bb6edd113fb6de","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2fed0338d4bb6edd113fb6de: start","job_status":"start","queue_duration_s":0.489289,"scheduling_latency_s":0.489305} gitlab # [ 222.757477] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.320Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2fed0338d4bb6edd113fb6de","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2fed0338d4bb6edd113fb6de: done: 0.014567 sec","job_status":"done","queue_duration_s":0.489289,"scheduling_latency_s":0.489305,"redis_calls":13,"redis_duration_s":0.001898,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00065,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001248,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009849,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014567,"completed_at":"2026-10-02T16:21:05.320Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 222.797694] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.321Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7fcfafc9f80fd33a9ec010e","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7fcfafc9f80fd33a9ec010e: start","job_status":"start","queue_duration_s":0.505056,"scheduling_latency_s":0.505071} gitlab # [ 222.810610] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.335Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7fcfafc9f80fd33a9ec010e","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7fcfafc9f80fd33a9ec010e: done: 0.014465 sec","job_status":"done","queue_duration_s":0.505056,"scheduling_latency_s":0.505071,"redis_calls":13,"redis_duration_s":0.001928,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000654,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001274,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009773,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014465,"completed_at":"2026-10-02T16:21:05.335Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 222.851643] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.337Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ffc1eae1c45f49764c7e799a","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ffc1eae1c45f49764c7e799a: start","job_status":"start","queue_duration_s":0.520691,"scheduling_latency_s":0.520706} gitlab # [ 222.864626] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.351Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ffc1eae1c45f49764c7e799a","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ffc1eae1c45f49764c7e799a: done: 0.014491 sec","job_status":"done","queue_duration_s":0.520691,"scheduling_latency_s":0.520706,"redis_calls":13,"redis_duration_s":0.0019820000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000691,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001291,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009109,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014491,"completed_at":"2026-10-02T16:21:05.351Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 222.905058] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.353Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3da457705c71f9874797fb73","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3da457705c71f9874797fb73: start","job_status":"start","queue_duration_s":0.536988,"scheduling_latency_s":0.537004} gitlab # [ 222.917819] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.368Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3da457705c71f9874797fb73","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3da457705c71f9874797fb73: done: 0.014909 sec","job_status":"done","queue_duration_s":0.536988,"scheduling_latency_s":0.537004,"redis_calls":13,"redis_duration_s":0.002196,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000898,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001298,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009948,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014909,"completed_at":"2026-10-02T16:21:05.368Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 222.958027] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.369Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea977f6a32a417e131592baa","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea977f6a32a417e131592baa: start","job_status":"start","queue_duration_s":0.553011,"scheduling_latency_s":0.553026} gitlab # [ 222.970791] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.383Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea977f6a32a417e131592baa","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea977f6a32a417e131592baa: done: 0.014106 sec","job_status":"done","queue_duration_s":0.553011,"scheduling_latency_s":0.553026,"redis_calls":13,"redis_duration_s":0.0018250000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000643,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001182,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009518,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014106,"completed_at":"2026-10-02T16:21:05.383Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 223.011225] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.384Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af8e147b2617de2467ac726f","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af8e147b2617de2467ac726f: start","job_status":"start","queue_duration_s":0.568322,"scheduling_latency_s":0.568338} gitlab # [ 223.024972] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.399Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af8e147b2617de2467ac726f","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af8e147b2617de2467ac726f: done: 0.014854 sec","job_status":"done","queue_duration_s":0.568322,"scheduling_latency_s":0.568338,"redis_calls":13,"redis_duration_s":0.001866,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000666,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0012,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010206,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014854,"completed_at":"2026-10-02T16:21:05.399Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 223.065034] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.400Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e886cf227438ee5be1c4c41c","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e886cf227438ee5be1c4c41c: start","job_status":"start","queue_duration_s":0.584364,"scheduling_latency_s":0.58438} gitlab # [ 223.077855] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.417Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e886cf227438ee5be1c4c41c","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e886cf227438ee5be1c4c41c: done: 0.016371 sec","job_status":"done","queue_duration_s":0.584364,"scheduling_latency_s":0.58438,"redis_calls":13,"redis_duration_s":0.002007,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00068,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001327,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008765,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016371,"completed_at":"2026-10-02T16:21:05.417Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 223.118388] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.418Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"01bff4dc2fb9cf9568d9c533","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-01bff4dc2fb9cf9568d9c533: start","job_status":"start","queue_duration_s":0.602014,"scheduling_latency_s":0.60203} gitlab # [ 223.131001] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.433Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"01bff4dc2fb9cf9568d9c533","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-01bff4dc2fb9cf9568d9c533: done: 0.014944 sec","job_status":"done","queue_duration_s":0.602014,"scheduling_latency_s":0.60203,"redis_calls":13,"redis_duration_s":0.0018509999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000654,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001197,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010193,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014944,"completed_at":"2026-10-02T16:21:05.433Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 223.171682] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.434Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e72d5131d16d49666084a6ee","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e72d5131d16d49666084a6ee: start","job_status":"start","queue_duration_s":0.618047,"scheduling_latency_s":0.618062} gitlab # [ 223.184500] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.449Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e72d5131d16d49666084a6ee","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e72d5131d16d49666084a6ee: done: 0.014567 sec","job_status":"done","queue_duration_s":0.618047,"scheduling_latency_s":0.618062,"redis_calls":13,"redis_duration_s":0.001909,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000667,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001242,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009938,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014567,"completed_at":"2026-10-02T16:21:05.448Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 223.225744] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.450Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54bd70ac19cc41e6e4d81653","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54bd70ac19cc41e6e4d81653: start","job_status":"start","queue_duration_s":0.633693,"scheduling_latency_s":0.633708} gitlab # [ 223.238519] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.463Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54bd70ac19cc41e6e4d81653","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54bd70ac19cc41e6e4d81653: done: 0.0137 sec","job_status":"done","queue_duration_s":0.633693,"scheduling_latency_s":0.633708,"redis_calls":13,"redis_duration_s":0.001775,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00066,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001115,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008705,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0137,"completed_at":"2026-10-02T16:21:05.463Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 223.278764] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.465Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56684332bcc443b1f6f010a0","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56684332bcc443b1f6f010a0: start","job_status":"start","queue_duration_s":0.649348,"scheduling_latency_s":0.649364} gitlab # [ 223.291704] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.479Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56684332bcc443b1f6f010a0","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56684332bcc443b1f6f010a0: done: 0.014106 sec","job_status":"done","queue_duration_s":0.649348,"scheduling_latency_s":0.649364,"redis_calls":13,"redis_duration_s":0.00182,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000684,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001136,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00941,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014106,"completed_at":"2026-10-02T16:21:05.479Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 223.332275] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.481Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64e86e4a31c6e7ca4b8265fd","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64e86e4a31c6e7ca4b8265fd: start","job_status":"start","queue_duration_s":0.664597,"scheduling_latency_s":0.664612} gitlab # [ 223.345042] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.500Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64e86e4a31c6e7ca4b8265fd","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64e86e4a31c6e7ca4b8265fd: done: 0.019352 sec","job_status":"done","queue_duration_s":0.664597,"scheduling_latency_s":0.664612,"redis_calls":13,"redis_duration_s":0.005882,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000925,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004957,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010976,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019352,"completed_at":"2026-10-02T16:21:05.500Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 223.384958] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.502Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a1cfc9410df943cadbdfe75","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a1cfc9410df943cadbdfe75: start","job_status":"start","queue_duration_s":0.68561,"scheduling_latency_s":0.685627} gitlab # [ 223.397718] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.519Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a1cfc9410df943cadbdfe75","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a1cfc9410df943cadbdfe75: done: 0.017889 sec","job_status":"done","queue_duration_s":0.68561,"scheduling_latency_s":0.685627,"redis_calls":13,"redis_duration_s":0.003429,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000844,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002585,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009626,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017889,"completed_at":"2026-10-02T16:21:05.519Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 223.439141] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.521Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75306342597b1d8af4b4b5da","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75306342597b1d8af4b4b5da: start","job_status":"start","queue_duration_s":0.704739,"scheduling_latency_s":0.704756} gitlab # [ 223.452223] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.534Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75306342597b1d8af4b4b5da","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75306342597b1d8af4b4b5da: done: 0.013789 sec","job_status":"done","queue_duration_s":0.704739,"scheduling_latency_s":0.704756,"redis_calls":13,"redis_duration_s":0.001901,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000681,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00122,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008803,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013789,"completed_at":"2026-10-02T16:21:05.534Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 223.492646] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.536Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d32bc40c37dc6e20df4f4f3","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d32bc40c37dc6e20df4f4f3: start","job_status":"start","queue_duration_s":0.719927,"scheduling_latency_s":0.719944} gitlab # [ 223.505564] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.549Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d32bc40c37dc6e20df4f4f3","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d32bc40c37dc6e20df4f4f3: done: 0.013546 sec","job_status":"done","queue_duration_s":0.719927,"scheduling_latency_s":0.719944,"redis_calls":13,"redis_duration_s":0.001899,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000644,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001255,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008986,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013546,"completed_at":"2026-10-02T16:21:05.549Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 223.545667] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.551Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1552434be6beebce727ab2f8","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1552434be6beebce727ab2f8: start","job_status":"start","queue_duration_s":0.734629,"scheduling_latency_s":0.734644} gitlab # [ 223.558509] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.564Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1552434be6beebce727ab2f8","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1552434be6beebce727ab2f8: done: 0.013835 sec","job_status":"done","queue_duration_s":0.734629,"scheduling_latency_s":0.734644,"redis_calls":13,"redis_duration_s":0.001963,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000722,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001241,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009197,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013835,"completed_at":"2026-10-02T16:21:05.564Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 223.598846] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.565Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27e69d549342de7ddf9533d5","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27e69d549342de7ddf9533d5: start","job_status":"start","queue_duration_s":0.749541,"scheduling_latency_s":0.749556} gitlab # [ 223.611510] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.579Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27e69d549342de7ddf9533d5","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27e69d549342de7ddf9533d5: done: 0.013655 sec","job_status":"done","queue_duration_s":0.749541,"scheduling_latency_s":0.749556,"redis_calls":13,"redis_duration_s":0.00183,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000632,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001198,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008877,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013655,"completed_at":"2026-10-02T16:21:05.579Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 223.652851] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.580Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3aa600e014b4eea033783e3","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3aa600e014b4eea033783e3: start","job_status":"start","queue_duration_s":0.764334,"scheduling_latency_s":0.76435} gitlab # [ 223.665751] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.594Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3aa600e014b4eea033783e3","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3aa600e014b4eea033783e3: done: 0.014191 sec","job_status":"done","queue_duration_s":0.764334,"scheduling_latency_s":0.76435,"redis_calls":13,"redis_duration_s":0.002052,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000714,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001338,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008984,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014191,"completed_at":"2026-10-02T16:21:05.594Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 223.706220] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.596Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65d8b81ce0194c806ea01862","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65d8b81ce0194c806ea01862: start","job_status":"start","queue_duration_s":0.779683,"scheduling_latency_s":0.779698} gitlab # [ 223.719182] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.609Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65d8b81ce0194c806ea01862","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65d8b81ce0194c806ea01862: done: 0.013703 sec","job_status":"done","queue_duration_s":0.779683,"scheduling_latency_s":0.779698,"redis_calls":13,"redis_duration_s":0.001853,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000632,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001221,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009156,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013703,"completed_at":"2026-10-02T16:21:05.609Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 223.759690] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.610Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8ec104de0d2fc03a35443cf","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8ec104de0d2fc03a35443cf: start","job_status":"start","queue_duration_s":0.794546,"scheduling_latency_s":0.794561} gitlab # [ 223.772765] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.624Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8ec104de0d2fc03a35443cf","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8ec104de0d2fc03a35443cf: done: 0.013843 sec","job_status":"done","queue_duration_s":0.794546,"scheduling_latency_s":0.794561,"redis_calls":13,"redis_duration_s":0.0019890000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000709,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00128,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009169,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013843,"completed_at":"2026-10-02T16:21:05.624Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 223.813310] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.625Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee82cb65abb54bdf145e2c8c","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee82cb65abb54bdf145e2c8c: start","job_status":"start","queue_duration_s":0.80956,"scheduling_latency_s":0.809576} gitlab # [ 223.826252] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.639Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee82cb65abb54bdf145e2c8c","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee82cb65abb54bdf145e2c8c: done: 0.013651 sec","job_status":"done","queue_duration_s":0.80956,"scheduling_latency_s":0.809576,"redis_calls":13,"redis_duration_s":0.001807,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000628,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001179,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009107,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013651,"completed_at":"2026-10-02T16:21:05.639Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 223.867465] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.640Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73ac61f093e52cf24020febd","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73ac61f093e52cf24020febd: start","job_status":"start","queue_duration_s":0.824339,"scheduling_latency_s":0.824354} gitlab # [ 223.880466] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.654Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73ac61f093e52cf24020febd","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73ac61f093e52cf24020febd: done: 0.013559 sec","job_status":"done","queue_duration_s":0.824339,"scheduling_latency_s":0.824354,"redis_calls":13,"redis_duration_s":0.0018510000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00062,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001231,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008744,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013559,"completed_at":"2026-10-02T16:21:05.654Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 223.921033] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.655Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0da7b8af62ad5c323ae88548","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0da7b8af62ad5c323ae88548: start","job_status":"start","queue_duration_s":0.839398,"scheduling_latency_s":0.839412} gitlab # [ 223.933838] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.669Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0da7b8af62ad5c323ae88548","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0da7b8af62ad5c323ae88548: done: 0.014015 sec","job_status":"done","queue_duration_s":0.839398,"scheduling_latency_s":0.839412,"redis_calls":13,"redis_duration_s":0.0019979999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000686,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001312,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00923,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014015,"completed_at":"2026-10-02T16:21:05.669Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 223.974242] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.671Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"da133e0ec65660932847ecec","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-da133e0ec65660932847ecec: start","job_status":"start","queue_duration_s":0.854605,"scheduling_latency_s":0.854621} gitlab # [ 223.987103] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.684Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"da133e0ec65660932847ecec","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-da133e0ec65660932847ecec: done: 0.013649 sec","job_status":"done","queue_duration_s":0.854605,"scheduling_latency_s":0.854621,"redis_calls":13,"redis_duration_s":0.001871,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000632,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001239,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009079,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013649,"completed_at":"2026-10-02T16:21:05.684Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 224.027564] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.685Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d301000caaf67296a28747eb","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d301000caaf67296a28747eb: start","job_status":"start","queue_duration_s":0.869377,"scheduling_latency_s":0.869393} gitlab # [ 224.041399] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.699Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d301000caaf67296a28747eb","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d301000caaf67296a28747eb: done: 0.013684 sec","job_status":"done","queue_duration_s":0.869377,"scheduling_latency_s":0.869393,"redis_calls":13,"redis_duration_s":0.0018809999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000648,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001233,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009123,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013684,"completed_at":"2026-10-02T16:21:05.699Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 224.081650] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.700Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dd5136f2af2a632b9016d1bd","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dd5136f2af2a632b9016d1bd: start","job_status":"start","queue_duration_s":0.884215,"scheduling_latency_s":0.884231} gitlab # [ 224.094411] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.714Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dd5136f2af2a632b9016d1bd","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dd5136f2af2a632b9016d1bd: done: 0.013414 sec","job_status":"done","queue_duration_s":0.884215,"scheduling_latency_s":0.884231,"redis_calls":13,"redis_duration_s":0.001795,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000621,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001174,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00875,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013414,"completed_at":"2026-10-02T16:21:05.713Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 224.134730] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.715Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d69ee3102be049169a2a7b5","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d69ee3102be049169a2a7b5: start","job_status":"start","queue_duration_s":0.899006,"scheduling_latency_s":0.899023} gitlab # [ 224.147647] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.729Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d69ee3102be049169a2a7b5","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d69ee3102be049169a2a7b5: done: 0.013787 sec","job_status":"done","queue_duration_s":0.899006,"scheduling_latency_s":0.899023,"redis_calls":13,"redis_duration_s":0.001904,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000657,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001247,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009083,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013787,"completed_at":"2026-10-02T16:21:05.729Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 224.188047] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.730Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d89ea6cc7c046cc48360bc1","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d89ea6cc7c046cc48360bc1: start","job_status":"start","queue_duration_s":0.913913,"scheduling_latency_s":0.913928} gitlab # [ 224.200651] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.743Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d89ea6cc7c046cc48360bc1","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d89ea6cc7c046cc48360bc1: done: 0.013573 sec","job_status":"done","queue_duration_s":0.913913,"scheduling_latency_s":0.913928,"redis_calls":13,"redis_duration_s":0.0017959999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000625,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001171,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009095,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013573,"completed_at":"2026-10-02T16:21:05.743Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 224.240994] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.745Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16e1f43c536b76baaf96e60d","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16e1f43c536b76baaf96e60d: start","job_status":"start","queue_duration_s":0.928734,"scheduling_latency_s":0.92875} gitlab # [ 224.254808] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.758Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16e1f43c536b76baaf96e60d","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16e1f43c536b76baaf96e60d: done: 0.01351 sec","job_status":"done","queue_duration_s":0.928734,"scheduling_latency_s":0.92875,"redis_calls":13,"redis_duration_s":0.001841,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000637,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001204,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008942,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01351,"completed_at":"2026-10-02T16:21:05.758Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 224.295032] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.759Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"05b7197a3a8814aa4793e0c6","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-05b7197a3a8814aa4793e0c6: start","job_status":"start","queue_duration_s":0.943374,"scheduling_latency_s":0.94339} gitlab # [ 224.307912] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.774Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"05b7197a3a8814aa4793e0c6","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-05b7197a3a8814aa4793e0c6: done: 0.014281 sec","job_status":"done","queue_duration_s":0.943374,"scheduling_latency_s":0.94339,"redis_calls":13,"redis_duration_s":0.002114,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000648,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001466,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009123,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014281,"completed_at":"2026-10-02T16:21:05.774Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 224.348275] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.775Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbcfc81ba2434cd5cef9d981","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbcfc81ba2434cd5cef9d981: start","job_status":"start","queue_duration_s":0.958767,"scheduling_latency_s":0.958782} gitlab # [ 224.361058] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.789Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbcfc81ba2434cd5cef9d981","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbcfc81ba2434cd5cef9d981: done: 0.014057 sec","job_status":"done","queue_duration_s":0.958767,"scheduling_latency_s":0.958782,"redis_calls":13,"redis_duration_s":0.00202,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000677,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001343,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009365,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014057,"completed_at":"2026-10-02T16:21:05.789Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 224.401160] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.790Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34d3d446039f720f03b0bd8f","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34d3d446039f720f03b0bd8f: start","job_status":"start","queue_duration_s":0.973966,"scheduling_latency_s":0.973995} gitlab # [ 224.414056] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.804Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34d3d446039f720f03b0bd8f","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34d3d446039f720f03b0bd8f: done: 0.013831 sec","job_status":"done","queue_duration_s":0.973966,"scheduling_latency_s":0.973995,"redis_calls":13,"redis_duration_s":0.001839,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000638,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001201,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009299,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013831,"completed_at":"2026-10-02T16:21:05.804Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 224.455612] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.805Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ead449a8aa7d8b423e9b59bb","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ead449a8aa7d8b423e9b59bb: start","job_status":"start","queue_duration_s":0.98896,"scheduling_latency_s":0.988976} gitlab # [ 224.468638] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.819Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ead449a8aa7d8b423e9b59bb","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ead449a8aa7d8b423e9b59bb: done: 0.013824 sec","job_status":"done","queue_duration_s":0.98896,"scheduling_latency_s":0.988976,"redis_calls":13,"redis_duration_s":0.001753,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000631,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001122,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00908,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013824,"completed_at":"2026-10-02T16:21:05.819Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 224.509028] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.820Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed8c4bde17bdd86fe8741f9e","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed8c4bde17bdd86fe8741f9e: start","job_status":"start","queue_duration_s":1.004012,"scheduling_latency_s":1.004028} gitlab # [ 224.521828] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.833Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed8c4bde17bdd86fe8741f9e","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed8c4bde17bdd86fe8741f9e: done: 0.013479 sec","job_status":"done","queue_duration_s":1.004012,"scheduling_latency_s":1.004028,"redis_calls":13,"redis_duration_s":0.001867,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000626,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001241,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008944,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013479,"completed_at":"2026-10-02T16:21:05.833Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 224.562031] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.835Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5ff24f4a35a4ca056d4c5cf","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5ff24f4a35a4ca056d4c5cf: start","job_status":"start","queue_duration_s":1.018614,"scheduling_latency_s":1.018629} gitlab # [ 224.574880] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.850Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5ff24f4a35a4ca056d4c5cf","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5ff24f4a35a4ca056d4c5cf: done: 0.015352 sec","job_status":"done","queue_duration_s":1.018614,"scheduling_latency_s":1.018629,"redis_calls":13,"redis_duration_s":0.001976,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00071,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001266,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010733,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015352,"completed_at":"2026-10-02T16:21:05.850Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 224.615282] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.851Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"898900dbd1aada108ee14b25","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-898900dbd1aada108ee14b25: start","job_status":"start","queue_duration_s":1.035012,"scheduling_latency_s":1.035027} gitlab # [ 224.628067] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.866Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"898900dbd1aada108ee14b25","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-898900dbd1aada108ee14b25: done: 0.014898 sec","job_status":"done","queue_duration_s":1.035012,"scheduling_latency_s":1.035027,"redis_calls":13,"redis_duration_s":0.001871,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000632,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001239,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014898,"completed_at":"2026-10-02T16:21:05.866Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 224.669309] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.867Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c848b7f804e32c6fc49b8ec","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c848b7f804e32c6fc49b8ec: start","job_status":"start","queue_duration_s":1.051108,"scheduling_latency_s":1.051124} gitlab # [ 224.682024] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.882Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c848b7f804e32c6fc49b8ec","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c848b7f804e32c6fc49b8ec: done: 0.015254 sec","job_status":"done","queue_duration_s":1.051108,"scheduling_latency_s":1.051124,"redis_calls":13,"redis_duration_s":0.001921,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000636,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001285,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010083,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015254,"completed_at":"2026-10-02T16:21:05.882Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 224.722709] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.883Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"10197f1990b935b7d3e81f58","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-10197f1990b935b7d3e81f58: start","job_status":"start","queue_duration_s":1.067501,"scheduling_latency_s":1.067516} gitlab # [ 224.735651] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.899Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"10197f1990b935b7d3e81f58","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-10197f1990b935b7d3e81f58: done: 0.01574 sec","job_status":"done","queue_duration_s":1.067501,"scheduling_latency_s":1.067516,"redis_calls":13,"redis_duration_s":0.00285,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001756,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009973,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01574,"completed_at":"2026-10-02T16:21:05.899Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 224.776132] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.900Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"680084d0d599b31ce0c90228","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-680084d0d599b31ce0c90228: start","job_status":"start","queue_duration_s":1.084403,"scheduling_latency_s":1.084419} gitlab # [ 224.788797] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.916Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"680084d0d599b31ce0c90228","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-680084d0d599b31ce0c90228: done: 0.015331 sec","job_status":"done","queue_duration_s":1.084403,"scheduling_latency_s":1.084419,"redis_calls":13,"redis_duration_s":0.001912,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000662,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00125,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009862,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015331,"completed_at":"2026-10-02T16:21:05.916Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 224.829249] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.917Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84e4aea60636e42126a8b330","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84e4aea60636e42126a8b330: start","job_status":"start","queue_duration_s":1.100907,"scheduling_latency_s":1.100923} gitlab # [ 224.849156] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.932Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84e4aea60636e42126a8b330","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84e4aea60636e42126a8b330: done: 0.014854 sec","job_status":"done","queue_duration_s":1.100907,"scheduling_latency_s":1.100923,"redis_calls":13,"redis_duration_s":0.001883,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000663,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00122,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009826,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014854,"completed_at":"2026-10-02T16:21:05.932Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 224.909692] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.933Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc02efa24ee73ca66f44800a","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc02efa24ee73ca66f44800a: start","job_status":"start","queue_duration_s":1.117298,"scheduling_latency_s":1.117315} gitlab # [ 224.924571] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.948Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc02efa24ee73ca66f44800a","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc02efa24ee73ca66f44800a: done: 0.014883 sec","job_status":"done","queue_duration_s":1.117298,"scheduling_latency_s":1.117315,"redis_calls":13,"redis_duration_s":0.0018510000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00064,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001211,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00974,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014883,"completed_at":"2026-10-02T16:21:05.948Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 224.965773] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.950Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4da5d9abac2e9506065098d5","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4da5d9abac2e9506065098d5: start","job_status":"start","queue_duration_s":1.13379,"scheduling_latency_s":1.133807} gitlab # [ 224.978462] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.965Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4da5d9abac2e9506065098d5","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4da5d9abac2e9506065098d5: done: 0.015062 sec","job_status":"done","queue_duration_s":1.13379,"scheduling_latency_s":1.133807,"redis_calls":13,"redis_duration_s":0.001875,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000728,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001147,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010381,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015062,"completed_at":"2026-10-02T16:21:05.965Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 225.018957] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.966Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21ef6ae796c27aab7203b71a","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21ef6ae796c27aab7203b71a: start","job_status":"start","queue_duration_s":1.150373,"scheduling_latency_s":1.150388} gitlab # [ 225.031793] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.981Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21ef6ae796c27aab7203b71a","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21ef6ae796c27aab7203b71a: done: 0.01513 sec","job_status":"done","queue_duration_s":1.150373,"scheduling_latency_s":1.150388,"redis_calls":13,"redis_duration_s":0.002232,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000932,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0013,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010204,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01513,"completed_at":"2026-10-02T16:21:05.981Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 225.073096] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.983Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"920b5d2574248331130b17e7","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-920b5d2574248331130b17e7: start","job_status":"start","queue_duration_s":1.167092,"scheduling_latency_s":1.167108} gitlab # [ 225.086007] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:05.998Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"920b5d2574248331130b17e7","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-920b5d2574248331130b17e7: done: 0.014972 sec","job_status":"done","queue_duration_s":1.167092,"scheduling_latency_s":1.167108,"redis_calls":13,"redis_duration_s":0.0019979999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000685,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001313,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010277,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014972,"completed_at":"2026-10-02T16:21:05.998Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 225.126376] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:06.000Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"633f8c9326b6c60d78a5953f","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-633f8c9326b6c60d78a5953f: start","job_status":"start","queue_duration_s":1.183665,"scheduling_latency_s":1.183682} gitlab # [ 225.139172] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:06.015Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"633f8c9326b6c60d78a5953f","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-633f8c9326b6c60d78a5953f: done: 0.014946 sec","job_status":"done","queue_duration_s":1.183665,"scheduling_latency_s":1.183682,"redis_calls":13,"redis_duration_s":0.0019360000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000684,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001252,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01028,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014946,"completed_at":"2026-10-02T16:21:06.014Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 225.179531] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:06.016Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c4800c8f4c46108fe53ff3d","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c4800c8f4c46108fe53ff3d: start","job_status":"start","queue_duration_s":1.19977,"scheduling_latency_s":1.199785} gitlab # [ 225.192314] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:06.031Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c4800c8f4c46108fe53ff3d","created_at":"2026-10-02T16:21:04.739Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-abde5133f4b74b68","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7c75446e5b8e4048c222a162663e93d5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:04.816Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c4800c8f4c46108fe53ff3d: done: 0.015198 sec","job_status":"done","queue_duration_s":1.19977,"scheduling_latency_s":1.199785,"redis_calls":13,"redis_duration_s":0.001787,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000629,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001158,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010076,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015198,"completed_at":"2026-10-02T16:21:06.031Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 225.232471] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:06.032Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a8df77c0a6881bb908d5fddf","created_at":"2026-10-02T16:21:05.142Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-9d8adc8501fe4a99","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"74db2be9de1f7263301e7b2798c88660","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:05.155Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a8df77c0a6881bb908d5fddf: start","job_status":"start","queue_duration_s":0.876608,"scheduling_latency_s":0.876624} gitlab # [ 225.245040] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:06.051Z","message":"No existing merge request to be cleaned up."} gitlab # [ 225.246790] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:06.066Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a8df77c0a6881bb908d5fddf","created_at":"2026-10-02T16:21:05.142Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-9d8adc8501fe4a99","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"74db2be9de1f7263301e7b2798c88660","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:05.155Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a8df77c0a6881bb908d5fddf: done: 0.033582 sec","job_status":"done","queue_duration_s":0.876608,"scheduling_latency_s":0.876624,"redis_calls":13,"redis_duration_s":0.002784,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000805,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000886,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020931,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002795,"duration_s":0.033582,"completed_at":"2026-10-02T16:21:06.066Z","load_balancing_strategy":"primary","db_duration_s":0.004404,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 225.289009] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:06.067Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c1082aaa7626baf172f52984","created_at":"2026-10-02T16:21:05.142Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-9d8adc8501fe4a99","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"74db2be9de1f7263301e7b2798c88660","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:05.155Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c1082aaa7626baf172f52984: start","job_status":"start","queue_duration_s":0.911818,"scheduling_latency_s":0.911835} gitlab # [ 225.301721] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:06.081Z","message":"No existing merge request to be cleaned up."} gitlab # [ 225.303426] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:06.090Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c1082aaa7626baf172f52984","created_at":"2026-10-02T16:21:05.142Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-9d8adc8501fe4a99","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"74db2be9de1f7263301e7b2798c88660","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:05.155Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c1082aaa7626baf172f52984: done: 0.02279 sec","job_status":"done","queue_duration_s":0.911818,"scheduling_latency_s":0.911835,"redis_calls":10,"redis_duration_s":0.001843,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000742,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001101,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01583,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02279,"completed_at":"2026-10-02T16:21:06.090Z","load_balancing_strategy":"primary","db_duration_s":0.001934,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 225.343747] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:06.091Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fe69631c52f639032274b164","created_at":"2026-10-02T16:21:05.142Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-9d8adc8501fe4a99","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"74db2be9de1f7263301e7b2798c88660","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:05.155Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fe69631c52f639032274b164: start","job_status":"start","queue_duration_s":0.935959,"scheduling_latency_s":0.935976} gitlab # [ 225.356939] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:06.103Z","message":"No existing merge request to be cleaned up."} gitlab # [ 225.358727] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:06.111Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fe69631c52f639032274b164","created_at":"2026-10-02T16:21:05.142Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-9d8adc8501fe4a99","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"74db2be9de1f7263301e7b2798c88660","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:05.155Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fe69631c52f639032274b164: done: 0.019734 sec","job_status":"done","queue_duration_s":0.935959,"scheduling_latency_s":0.935976,"redis_calls":10,"redis_duration_s":0.001865,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000761,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001104,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013142,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019734,"completed_at":"2026-10-02T16:21:06.111Z","load_balancing_strategy":"primary","db_duration_s":0.001827,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 225.398717] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:06.112Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"890f391c03be7995adfe094a","created_at":"2026-10-02T16:21:05.142Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-9d8adc8501fe4a99","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"74db2be9de1f7263301e7b2798c88660","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:05.155Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-890f391c03be7995adfe094a: start","job_status":"start","queue_duration_s":0.957001,"scheduling_latency_s":0.957017} gitlab # [ 225.412005] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:06.123Z","message":"No existing merge request to be cleaned up."} gitlab # [ 225.413733] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:21:06.131Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"890f391c03be7995adfe094a","created_at":"2026-10-02T16:21:05.142Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-9d8adc8501fe4a99","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"74db2be9de1f7263301e7b2798c88660","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:21:05.155Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-890f391c03be7995adfe094a: done: 0.018928 sec","job_status":"done","queue_duration_s":0.957001,"scheduling_latency_s":0.957017,"redis_calls":10,"redis_duration_s":0.0018440000000000002,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000754,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00109,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012849,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018928,"completed_at":"2026-10-02T16:21:06.131Z","load_balancing_strategy":"primary","db_duration_s":0.001716,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 261.975484] setup-gitlab-start[2985]: W, [2026-10-02T16:21:44.825088 #2985] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 278.923922] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:01.773Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"442be5efe902b1eca428a27d","created_at":"2026-10-02T16:22:01.760Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bf6ade7bfe4b9764b304dd346db60a27","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:01.770Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-442be5efe902b1eca428a27d: start","job_status":"start","queue_duration_s":0.002156,"scheduling_latency_s":0.002195} gitlab # [ 278.955118] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:01.805Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 278.989239] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:01.838Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"442be5efe902b1eca428a27d","created_at":"2026-10-02T16:22:01.760Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bf6ade7bfe4b9764b304dd346db60a27","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:01.770Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-442be5efe902b1eca428a27d: done: 0.065907 sec","job_status":"done","queue_duration_s":0.002156,"scheduling_latency_s":0.002195,"redis_calls":14,"redis_duration_s":0.021246,"redis_read_bytes":777,"redis_write_bytes":1225,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.012855,"redis_feature_flag_read_bytes":768,"redis_feature_flag_write_bytes":400,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005429,"redis_queues_metadata_read_bytes":8,"redis_queues_metadata_write_bytes":561,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002962,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022755,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.065907,"completed_at":"2026-10-02T16:22:01.838Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.052958] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:01.841Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"de2c2bef0dbc5b47eca6bf13","created_at":"2026-10-02T16:22:01.785Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"732f3caec999b15fd3f2ce5d8c9e8958","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:01.798Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-de2c2bef0dbc5b47eca6bf13: start","job_status":"start","queue_duration_s":0.042725,"scheduling_latency_s":0.042754} gitlab # [ 279.090931] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:01.940Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"de2c2bef0dbc5b47eca6bf13","created_at":"2026-10-02T16:22:01.785Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"732f3caec999b15fd3f2ce5d8c9e8958","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:01.798Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-de2c2bef0dbc5b47eca6bf13: done: 0.099237 sec","job_status":"done","queue_duration_s":0.042725,"scheduling_latency_s":0.042754,"redis_calls":11,"redis_duration_s":0.008008,"redis_read_bytes":136,"redis_write_bytes":1288,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000197,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":49,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005945,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001866,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.06,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.06,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023862,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.06499118,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.099237,"completed_at":"2026-10-02T16:22:01.940Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.059699,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 279.146862] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:01.942Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"e0725d0c23ff1b35b007cc2c","created_at":"2026-10-02T16:22:01.811Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1b439048b3bd08808f2d43c874185b8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:01.820Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-e0725d0c23ff1b35b007cc2c: start","job_status":"start","queue_duration_s":0.121733,"scheduling_latency_s":0.121762} gitlab # [ 279.163082] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:01.956Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 279.166884] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:01.965Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"e0725d0c23ff1b35b007cc2c","created_at":"2026-10-02T16:22:01.811Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1b439048b3bd08808f2d43c874185b8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:01.820Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-e0725d0c23ff1b35b007cc2c: done: 0.022979 sec","job_status":"done","queue_duration_s":0.121733,"scheduling_latency_s":0.121762,"redis_calls":7,"redis_duration_s":0.0015760000000000001,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00109,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000486,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013054,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022979,"completed_at":"2026-10-02T16:22:01.965Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.209357] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:01.967Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"ac26885e151658065e11fd8f","created_at":"2026-10-02T16:22:01.832Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"31cf93cea3bf0339d79add42e01d9610","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:01.835Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-ac26885e151658065e11fd8f: start","job_status":"start","queue_duration_s":0.131895,"scheduling_latency_s":0.13192} gitlab # [ 279.228148] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.077Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"ac26885e151658065e11fd8f","created_at":"2026-10-02T16:22:01.832Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"31cf93cea3bf0339d79add42e01d9610","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:01.835Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-ac26885e151658065e11fd8f: done: 0.110482 sec","job_status":"done","queue_duration_s":0.131895,"scheduling_latency_s":0.13192,"redis_calls":7,"redis_duration_s":0.003517,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00255,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000967,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.066,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.066,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016882,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.110482,"completed_at":"2026-10-02T16:22:02.077Z","load_balancing_strategy":"primary","db_duration_s":0.065533,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.270702] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.085Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"4556d6e1cb08d1ab4fec9350","created_at":"2026-10-02T16:22:01.858Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"73aa48d5e56a1a662e79acb7dcc0a7af","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:01.862Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-4556d6e1cb08d1ab4fec9350: start","job_status":"start","queue_duration_s":0.223323,"scheduling_latency_s":0.223353} gitlab # [ 279.285588] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.130Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 279.297097] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.146Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"4556d6e1cb08d1ab4fec9350","created_at":"2026-10-02T16:22:01.858Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"73aa48d5e56a1a662e79acb7dcc0a7af","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:01.862Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-4556d6e1cb08d1ab4fec9350: done: 0.061576 sec","job_status":"done","queue_duration_s":0.223323,"scheduling_latency_s":0.223353,"redis_calls":7,"redis_duration_s":0.011273,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008193,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00308,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014606,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.061576,"completed_at":"2026-10-02T16:22:02.146Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.341258] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.191Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"cf95a7ff8a8504fba7408520","created_at":"2026-10-02T16:22:01.865Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f7f8df3cf072e1d6547f9c83304aa369","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:01.868Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-cf95a7ff8a8504fba7408520: start","job_status":"start","queue_duration_s":0.322763,"scheduling_latency_s":0.322789} gitlab # [ 279.388146] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.235Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"cf95a7ff8a8504fba7408520","created_at":"2026-10-02T16:22:01.865Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f7f8df3cf072e1d6547f9c83304aa369","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:01.868Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-cf95a7ff8a8504fba7408520: done: 0.04411 sec","job_status":"done","queue_duration_s":0.322763,"scheduling_latency_s":0.322789,"redis_calls":8,"redis_duration_s":0.013882,"redis_read_bytes":133,"redis_write_bytes":846,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.00107,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":70,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.010794,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002018,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016013,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04411,"completed_at":"2026-10-02T16:22:02.235Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.433598] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.257Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"351b5a0d1d04c4771b6fc76c","created_at":"2026-10-02T16:22:01.872Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1e735819c18f9a1f0c2df1e4b44a657a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:01.875Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-351b5a0d1d04c4771b6fc76c: start","job_status":"start","queue_duration_s":0.381394,"scheduling_latency_s":0.381418} gitlab # [ 279.535653] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.385Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"351b5a0d1d04c4771b6fc76c","created_at":"2026-10-02T16:22:01.872Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1e735819c18f9a1f0c2df1e4b44a657a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:01.875Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-351b5a0d1d04c4771b6fc76c: done: 0.128278 sec","job_status":"done","queue_duration_s":0.381394,"scheduling_latency_s":0.381418,"redis_calls":9,"redis_duration_s":0.012637,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.010937,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.0017,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.02,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00095128,"exclusive_lock_hold_duration_s":0.08857398,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.128278,"completed_at":"2026-10-02T16:22:02.385Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.08618,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.602490] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.387Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"1fccef455f7269d98cc69cbd","created_at":"2026-10-02T16:22:01.906Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ec50079ecfb41cf9285b755d56edf35","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:01.914Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-1fccef455f7269d98cc69cbd: start","job_status":"start","queue_duration_s":0.472902,"scheduling_latency_s":0.472926} gitlab # [ 279.619128] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.419Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"1fccef455f7269d98cc69cbd","created_at":"2026-10-02T16:22:01.906Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ec50079ecfb41cf9285b755d56edf35","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:01.914Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-1fccef455f7269d98cc69cbd: done: 0.032635 sec","job_status":"done","queue_duration_s":0.472902,"scheduling_latency_s":0.472926,"redis_calls":7,"redis_duration_s":0.001735,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001221,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000514,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01634,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032635,"completed_at":"2026-10-02T16:22:02.419Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008597,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.661565] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.421Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"d8d4504adf8493a572af8351","created_at":"2026-10-02T16:22:01.921Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"35f8df0a677f35ec05201b9c26fb34dd","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:01.984Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-d8d4504adf8493a572af8351: start","job_status":"start","queue_duration_s":0.43673,"scheduling_latency_s":0.436754} gitlab # [ 279.676203] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.488Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"d8d4504adf8493a572af8351","created_at":"2026-10-02T16:22:01.921Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"35f8df0a677f35ec05201b9c26fb34dd","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:01.984Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-d8d4504adf8493a572af8351: done: 0.067394 sec","job_status":"done","queue_duration_s":0.43673,"scheduling_latency_s":0.436754,"redis_calls":28,"redis_duration_s":0.007749,"redis_read_bytes":36,"redis_write_bytes":10603,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000874,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":680,"redis_queues_calls":2,"redis_queues_duration_s":0.000476,"redis_queues_read_bytes":3,"redis_queues_write_bytes":8315,"redis_queues_metadata_calls":16,"redis_queues_metadata_duration_s":0.004722,"redis_queues_metadata_read_bytes":24,"redis_queues_metadata_write_bytes":1162,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001677,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":446,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.048804,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031354,"duration_s":0.067394,"completed_at":"2026-10-02T16:22:02.488Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004659,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.721094] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.490Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"cd69999ad69a3cdc23b6b7e6","created_at":"2026-10-02T16:22:01.988Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ef445f70c3a7044d2a3792199fe1562","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:01.992Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-cd69999ad69a3cdc23b6b7e6: start","job_status":"start","queue_duration_s":0.497619,"scheduling_latency_s":0.497638} gitlab # [ 279.735081] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.508Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"cd69999ad69a3cdc23b6b7e6","created_at":"2026-10-02T16:22:01.988Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ef445f70c3a7044d2a3792199fe1562","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:01.992Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-cd69999ad69a3cdc23b6b7e6: done: 0.018694 sec","job_status":"done","queue_duration_s":0.497619,"scheduling_latency_s":0.497638,"redis_calls":9,"redis_duration_s":0.0017929999999999999,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000932,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000861,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011723,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00058334,"exclusive_lock_hold_duration_s":0.00371192,"duration_s":0.018694,"completed_at":"2026-10-02T16:22:02.508Z","load_balancing_strategy":"primary","db_duration_s":0.00133,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.777128] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.510Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"ff1fd52493ff3f8ed07e03ac","created_at":"2026-10-02T16:22:02.004Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6c5fecc2977ac47d505a530bd59844aa","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T16:22:02.007Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-ff1fd52493ff3f8ed07e03ac: start","job_status":"start","queue_duration_s":0.502736,"scheduling_latency_s":0.502753} gitlab # [ 279.790982] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.518Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 279.794537] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.523Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"ff1fd52493ff3f8ed07e03ac","created_at":"2026-10-02T16:22:02.004Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6c5fecc2977ac47d505a530bd59844aa","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T16:22:02.007Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-ff1fd52493ff3f8ed07e03ac: done: 0.013507 sec","job_status":"done","queue_duration_s":0.502736,"scheduling_latency_s":0.502753,"redis_calls":7,"redis_duration_s":0.001197,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000856,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000341,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008616,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013507,"completed_at":"2026-10-02T16:22:02.523Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.835184] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.525Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"214c356f6a0779a85c2f933e","created_at":"2026-10-02T16:22:02.037Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c918156f5bd8b5129bf28952f823ee57","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.044Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-214c356f6a0779a85c2f933e: start","job_status":"start","queue_duration_s":0.48083,"scheduling_latency_s":0.480848} gitlab # [ 279.850442] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.540Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"214c356f6a0779a85c2f933e","created_at":"2026-10-02T16:22:02.037Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c918156f5bd8b5129bf28952f823ee57","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.044Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-214c356f6a0779a85c2f933e: done: 0.01506 sec","job_status":"done","queue_duration_s":0.48083,"scheduling_latency_s":0.480848,"redis_calls":7,"redis_duration_s":0.00156,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001238,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000322,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00879,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01506,"completed_at":"2026-10-02T16:22:02.540Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.892259] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.542Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"e5117b6ad614359e87572b98","created_at":"2026-10-02T16:22:02.059Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"227b1b7ee1d49e8728dde09a52e51163","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.067Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-e5117b6ad614359e87572b98: start","job_status":"start","queue_duration_s":0.474271,"scheduling_latency_s":0.474296} gitlab # [ 279.906633] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.563Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"e5117b6ad614359e87572b98","created_at":"2026-10-02T16:22:02.059Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"227b1b7ee1d49e8728dde09a52e51163","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.067Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-e5117b6ad614359e87572b98: done: 0.021228 sec","job_status":"done","queue_duration_s":0.474271,"scheduling_latency_s":0.474296,"redis_calls":7,"redis_duration_s":0.001618,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001108,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00051,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013742,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021228,"completed_at":"2026-10-02T16:22:02.563Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001167,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.948487] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.565Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"19a32fc0bf5df2c3f0ea45b1","created_at":"2026-10-02T16:22:02.076Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"82a9407a197169d825ebead4d1e2dc53","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T16:22:02.082Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-19a32fc0bf5df2c3f0ea45b1: start","job_status":"start","queue_duration_s":0.483228,"scheduling_latency_s":0.483252} gitlab # [ 279.962024] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.595Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"19a32fc0bf5df2c3f0ea45b1","created_at":"2026-10-02T16:22:02.076Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"82a9407a197169d825ebead4d1e2dc53","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T16:22:02.082Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-19a32fc0bf5df2c3f0ea45b1: done: 0.029678 sec","job_status":"done","queue_duration_s":0.483228,"scheduling_latency_s":0.483252,"redis_calls":7,"redis_duration_s":0.002401,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001683,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000718,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021232,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029678,"completed_at":"2026-10-02T16:22:02.595Z","load_balancing_strategy":"primary","db_duration_s":0.001898,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.003600] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.597Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"e413f3898fc762f27f9ae7fb","created_at":"2026-10-02T16:22:02.103Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"02b6f56ae6ee99875bd768332142c0d8","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.111Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-e413f3898fc762f27f9ae7fb: start","job_status":"start","queue_duration_s":0.485701,"scheduling_latency_s":0.485724} gitlab # [ 280.018644] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.628Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"e413f3898fc762f27f9ae7fb","created_at":"2026-10-02T16:22:02.103Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"02b6f56ae6ee99875bd768332142c0d8","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.111Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-e413f3898fc762f27f9ae7fb: done: 0.030981 sec","job_status":"done","queue_duration_s":0.485701,"scheduling_latency_s":0.485724,"redis_calls":12,"redis_duration_s":0.002508,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000426,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001142,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00094,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018345,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0006293,"exclusive_lock_hold_duration_s":0.00610356,"duration_s":0.030981,"completed_at":"2026-10-02T16:22:02.628Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002787,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.062478] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.629Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"da0230d2e1c9eaad818a1522","created_at":"2026-10-02T16:22:02.130Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c3d6d1d351ed4803bee3bbbd4fd2c02c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.138Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-da0230d2e1c9eaad818a1522: start","job_status":"start","queue_duration_s":0.491574,"scheduling_latency_s":0.491596} gitlab # [ 280.077045] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.862Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"da0230d2e1c9eaad818a1522","created_at":"2026-10-02T16:22:02.130Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c3d6d1d351ed4803bee3bbbd4fd2c02c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.138Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-da0230d2e1c9eaad818a1522: done: 0.23238 sec","job_status":"done","queue_duration_s":0.491574,"scheduling_latency_s":0.491596,"redis_calls":11,"redis_duration_s":0.002801,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000417,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00141,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000974,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":130,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.031,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.190862,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0005701,"exclusive_lock_hold_duration_s":0.21196928,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.23238,"completed_at":"2026-10-02T16:22:02.862Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.048182,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.122319] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.863Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"22923d6b1347a868a61a9f10","created_at":"2026-10-02T16:22:02.190Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b9cedff1fd3d7b1e0b80118884e26500","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T16:22:02.195Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-22923d6b1347a868a61a9f10: start","job_status":"start","queue_duration_s":0.66803,"scheduling_latency_s":0.668047} gitlab # [ 280.135922] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.871Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 280.139947] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.876Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"22923d6b1347a868a61a9f10","created_at":"2026-10-02T16:22:02.190Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b9cedff1fd3d7b1e0b80118884e26500","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T16:22:02.195Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-22923d6b1347a868a61a9f10: done: 0.012643 sec","job_status":"done","queue_duration_s":0.66803,"scheduling_latency_s":0.668047,"redis_calls":7,"redis_duration_s":0.001216,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000884,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000332,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007751,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012643,"completed_at":"2026-10-02T16:22:02.876Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.181935] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.877Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"bc3c5c422f536461497337cf","created_at":"2026-10-02T16:22:02.218Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c597be973190e05cf9d0b49d41d88487","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.232Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-bc3c5c422f536461497337cf: start","job_status":"start","queue_duration_s":0.645436,"scheduling_latency_s":0.645452} gitlab # [ 280.195612] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.895Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"bc3c5c422f536461497337cf","created_at":"2026-10-02T16:22:02.218Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c597be973190e05cf9d0b49d41d88487","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.232Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-bc3c5c422f536461497337cf: done: 0.018168 sec","job_status":"done","queue_duration_s":0.645436,"scheduling_latency_s":0.645452,"redis_calls":12,"redis_duration_s":0.002208,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000444,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000365,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001029,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00037,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010884,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028122,"duration_s":0.018168,"completed_at":"2026-10-02T16:22:02.895Z","load_balancing_strategy":"primary","db_duration_s":0.000378,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.240451] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:02.897Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"c7b3b4f1bf34a80151701799","created_at":"2026-10-02T16:22:02.253Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"14bb142dbebf58e5106f8f8e9dbea841","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.259Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-c7b3b4f1bf34a80151701799: start","job_status":"start","queue_duration_s":0.638082,"scheduling_latency_s":0.638152} gitlab # [ 280.341726] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.191Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"c7b3b4f1bf34a80151701799","created_at":"2026-10-02T16:22:02.253Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"14bb142dbebf58e5106f8f8e9dbea841","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.259Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-c7b3b4f1bf34a80151701799: done: 0.294348 sec","job_status":"done","queue_duration_s":0.638082,"scheduling_latency_s":0.638152,"redis_calls":562,"redis_duration_s":0.053655,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000914,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.052741,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.273745,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.294348,"completed_at":"2026-10-02T16:22:03.191Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.398899] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.192Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"67bf015cca1f3cfd9c7a3e00","created_at":"2026-10-02T16:22:02.280Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6289b2e749443a1fc2f73d689978ea51","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.286Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-67bf015cca1f3cfd9c7a3e00: start","job_status":"start","queue_duration_s":0.906439,"scheduling_latency_s":0.906465} gitlab # [ 280.415682] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.211Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"67bf015cca1f3cfd9c7a3e00","created_at":"2026-10-02T16:22:02.280Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6289b2e749443a1fc2f73d689978ea51","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.286Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-67bf015cca1f3cfd9c7a3e00: done: 0.018793 sec","job_status":"done","queue_duration_s":0.906439,"scheduling_latency_s":0.906465,"redis_calls":9,"redis_duration_s":0.001458,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000734,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000724,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011151,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00051346,"exclusive_lock_hold_duration_s":0.00375266,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.018793,"completed_at":"2026-10-02T16:22:03.211Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001596,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.459204] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.212Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"d0086c30ef8092c1e2ab5216","created_at":"2026-10-02T16:22:02.298Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.301Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-d0086c30ef8092c1e2ab5216: start","job_status":"start","queue_duration_s":0.911688,"scheduling_latency_s":0.911708} gitlab # [ 280.482758] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.332Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"d0086c30ef8092c1e2ab5216","created_at":"2026-10-02T16:22:02.298Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.301Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-d0086c30ef8092c1e2ab5216: done: 0.119499 sec","job_status":"done","queue_duration_s":0.911688,"scheduling_latency_s":0.911708,"redis_calls":60,"redis_duration_s":0.010704,"redis_read_bytes":108,"redis_write_bytes":41109,"redis_queues_calls":2,"redis_queues_duration_s":0.000869,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.009337,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000498,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.100058,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.119499,"completed_at":"2026-10-02T16:22:03.332Z","load_balancing_strategy":"primary","db_duration_s":0.006495,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.525192] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.334Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"2488ba7321b7fe2ccbd2f4e2","created_at":"2026-10-02T16:22:02.312Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"63a571c0abb9cfae7a2972fa89756904","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.316Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-2488ba7321b7fe2ccbd2f4e2: start","job_status":"start","queue_duration_s":1.018008,"scheduling_latency_s":1.018033} gitlab # [ 280.539000] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.371Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"2488ba7321b7fe2ccbd2f4e2","created_at":"2026-10-02T16:22:02.312Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"63a571c0abb9cfae7a2972fa89756904","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.316Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-2488ba7321b7fe2ccbd2f4e2: done: 0.037326 sec","job_status":"done","queue_duration_s":1.018008,"scheduling_latency_s":1.018033,"redis_calls":14,"redis_duration_s":0.003358,"redis_read_bytes":16,"redis_write_bytes":4026,"redis_queues_calls":2,"redis_queues_duration_s":0.000276,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002347,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000735,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028531,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037326,"completed_at":"2026-10-02T16:22:03.371Z","load_balancing_strategy":"primary","db_duration_s":0.001797,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.582643] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.373Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"3f4462d01b339e207e3258e3","created_at":"2026-10-02T16:22:02.319Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9cf98c706b74c837b2d58234d3fdc632","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-3f4462d01b339e207e3258e3: start","job_status":"start","queue_duration_s":1.049472,"scheduling_latency_s":1.049497} gitlab # [ 280.597885] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.415Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"3f4462d01b339e207e3258e3","created_at":"2026-10-02T16:22:02.319Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9cf98c706b74c837b2d58234d3fdc632","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-3f4462d01b339e207e3258e3: done: 0.042445 sec","job_status":"done","queue_duration_s":1.049472,"scheduling_latency_s":1.049497,"redis_calls":9,"redis_duration_s":0.001935,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001019,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000916,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023949,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00070182,"exclusive_lock_hold_duration_s":0.01628248,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.042445,"completed_at":"2026-10-02T16:22:03.415Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013382,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.642249] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.417Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"48e75a48b4246d84248603a6","created_at":"2026-10-02T16:22:02.328Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c8001eb4a497e4db2ffb89b8efb672fd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.332Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-48e75a48b4246d84248603a6: start","job_status":"start","queue_duration_s":1.084889,"scheduling_latency_s":1.084918} gitlab # [ 280.657533] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.428Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 280.661055] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.435Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"48e75a48b4246d84248603a6","created_at":"2026-10-02T16:22:02.328Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c8001eb4a497e4db2ffb89b8efb672fd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.332Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-48e75a48b4246d84248603a6: done: 0.017619 sec","job_status":"done","queue_duration_s":1.084889,"scheduling_latency_s":1.084918,"redis_calls":7,"redis_duration_s":0.001364,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000982,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000382,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017619,"completed_at":"2026-10-02T16:22:03.434Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.702861] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.436Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"4fc4db1d72c53d161faa5ae8","created_at":"2026-10-02T16:22:02.341Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"da81ac204cc918bee717931a72bbe3ea","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.349Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-4fc4db1d72c53d161faa5ae8: start","job_status":"start","queue_duration_s":1.086919,"scheduling_latency_s":1.086942} gitlab # [ 280.718078] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.467Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"4fc4db1d72c53d161faa5ae8","created_at":"2026-10-02T16:22:02.341Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-edbe03ea7b3b45a7","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"da81ac204cc918bee717931a72bbe3ea","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.349Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-4fc4db1d72c53d161faa5ae8: done: 0.031171 sec","job_status":"done","queue_duration_s":1.086919,"scheduling_latency_s":1.086942,"redis_calls":10,"redis_duration_s":0.001609,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000324,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000939,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000346,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019982,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00030264,"duration_s":0.031171,"completed_at":"2026-10-02T16:22:03.467Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003513,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.761696] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.469Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"d3223e5a71ea6a95e5d47c17","created_at":"2026-10-02T16:22:02.449Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-6cde7b73556548bc","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"35f8df0a677f35ec05201b9c26fb34dd","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.479Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-d3223e5a71ea6a95e5d47c17: start","job_status":"start","queue_duration_s":0.989813,"scheduling_latency_s":0.989835} gitlab # [ 280.775510] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.515Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"d3223e5a71ea6a95e5d47c17","created_at":"2026-10-02T16:22:02.449Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-6cde7b73556548bc","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"35f8df0a677f35ec05201b9c26fb34dd","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.479Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-d3223e5a71ea6a95e5d47c17: done: 0.046249 sec","job_status":"done","queue_duration_s":0.989813,"scheduling_latency_s":0.989835,"redis_calls":16,"redis_duration_s":0.00262,"redis_read_bytes":14,"redis_write_bytes":1526,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000335,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000737,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":423,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001548,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":802,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026192,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003059,"duration_s":0.046249,"completed_at":"2026-10-02T16:22:03.515Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012398,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.823827] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.517Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"0d57c1b417d5330c0ab763dc","created_at":"2026-10-02T16:22:02.449Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-6cde7b73556548bc","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"35f8df0a677f35ec05201b9c26fb34dd","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.479Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-0d57c1b417d5330c0ab763dc: start","job_status":"start","queue_duration_s":1.03766,"scheduling_latency_s":1.037684} gitlab # [ 280.837861] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.536Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"0d57c1b417d5330c0ab763dc","created_at":"2026-10-02T16:22:02.449Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-6cde7b73556548bc","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"35f8df0a677f35ec05201b9c26fb34dd","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.479Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-0d57c1b417d5330c0ab763dc: done: 0.019153 sec","job_status":"done","queue_duration_s":1.03766,"scheduling_latency_s":1.037684,"redis_calls":13,"redis_duration_s":0.001931,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000663,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001268,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012171,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019153,"completed_at":"2026-10-02T16:22:03.536Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001219,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.878387] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.537Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"715d4e61e38aac71d0853bae","created_at":"2026-10-02T16:22:02.449Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-6cde7b73556548bc","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"35f8df0a677f35ec05201b9c26fb34dd","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.479Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-715d4e61e38aac71d0853bae: start","job_status":"start","queue_duration_s":1.058224,"scheduling_latency_s":1.058246} gitlab # [ 280.892226] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.556Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"715d4e61e38aac71d0853bae","created_at":"2026-10-02T16:22:02.449Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-6cde7b73556548bc","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"35f8df0a677f35ec05201b9c26fb34dd","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.479Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-715d4e61e38aac71d0853bae: done: 0.018608 sec","job_status":"done","queue_duration_s":1.058224,"scheduling_latency_s":1.058246,"redis_calls":13,"redis_duration_s":0.002205,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000663,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001542,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011856,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018608,"completed_at":"2026-10-02T16:22:03.556Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001137,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.933478] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.557Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"7d07f90ab2da9d023ce84805","created_at":"2026-10-02T16:22:02.449Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-6cde7b73556548bc","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"35f8df0a677f35ec05201b9c26fb34dd","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.479Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-7d07f90ab2da9d023ce84805: start","job_status":"start","queue_duration_s":1.078328,"scheduling_latency_s":1.078351} gitlab # [ 280.946933] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.574Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"7d07f90ab2da9d023ce84805","created_at":"2026-10-02T16:22:02.449Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-6cde7b73556548bc","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"35f8df0a677f35ec05201b9c26fb34dd","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.479Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-7d07f90ab2da9d023ce84805: done: 0.016903 sec","job_status":"done","queue_duration_s":1.078328,"scheduling_latency_s":1.078351,"redis_calls":13,"redis_duration_s":0.001637,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00056,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001077,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010821,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016903,"completed_at":"2026-10-02T16:22:03.574Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001033,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.988548] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.575Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"b9534f049e9565dd0c08a701","created_at":"2026-10-02T16:22:02.449Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-6cde7b73556548bc","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"35f8df0a677f35ec05201b9c26fb34dd","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.479Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-b9534f049e9565dd0c08a701: start","job_status":"start","queue_duration_s":1.096483,"scheduling_latency_s":1.096503} gitlab # [ 281.003360] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.592Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"b9534f049e9565dd0c08a701","created_at":"2026-10-02T16:22:02.449Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-6cde7b73556548bc","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"35f8df0a677f35ec05201b9c26fb34dd","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.479Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-b9534f049e9565dd0c08a701: done: 0.016593 sec","job_status":"done","queue_duration_s":1.096483,"scheduling_latency_s":1.096503,"redis_calls":13,"redis_duration_s":0.0015170000000000001,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000992,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010123,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016593,"completed_at":"2026-10-02T16:22:03.592Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001818,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.044602] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.593Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"b720b173d852807ae11337bc","created_at":"2026-10-02T16:22:02.449Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-6cde7b73556548bc","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"35f8df0a677f35ec05201b9c26fb34dd","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.479Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-b720b173d852807ae11337bc: start","job_status":"start","queue_duration_s":1.114423,"scheduling_latency_s":1.114444} gitlab # [ 281.058369] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.609Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"b720b173d852807ae11337bc","created_at":"2026-10-02T16:22:02.449Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-6cde7b73556548bc","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"35f8df0a677f35ec05201b9c26fb34dd","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.479Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-b720b173d852807ae11337bc: done: 0.015789 sec","job_status":"done","queue_duration_s":1.114423,"scheduling_latency_s":1.114444,"redis_calls":13,"redis_duration_s":0.001521,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000987,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015789,"completed_at":"2026-10-02T16:22:03.609Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001102,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.099492] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.610Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"c2841da629ceb859a73e9ab1","created_at":"2026-10-02T16:22:02.449Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-6cde7b73556548bc","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"35f8df0a677f35ec05201b9c26fb34dd","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.479Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-c2841da629ceb859a73e9ab1: start","job_status":"start","queue_duration_s":1.131474,"scheduling_latency_s":1.131494} gitlab # [ 281.113225] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.626Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"c2841da629ceb859a73e9ab1","created_at":"2026-10-02T16:22:02.449Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-6cde7b73556548bc","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"35f8df0a677f35ec05201b9c26fb34dd","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.479Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-c2841da629ceb859a73e9ab1: done: 0.015446 sec","job_status":"done","queue_duration_s":1.131474,"scheduling_latency_s":1.131494,"redis_calls":13,"redis_duration_s":0.001621,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000576,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001045,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015446,"completed_at":"2026-10-02T16:22:03.626Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0007,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.154233] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.627Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"4513ee1dc63c8727d3b2f74c","created_at":"2026-10-02T16:22:02.449Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-6cde7b73556548bc","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"35f8df0a677f35ec05201b9c26fb34dd","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.479Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-4513ee1dc63c8727d3b2f74c: start","job_status":"start","queue_duration_s":1.148211,"scheduling_latency_s":1.148232} gitlab # [ 281.168039] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.643Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"4513ee1dc63c8727d3b2f74c","created_at":"2026-10-02T16:22:02.449Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-6cde7b73556548bc","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"35f8df0a677f35ec05201b9c26fb34dd","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.479Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-4513ee1dc63c8727d3b2f74c: done: 0.015682 sec","job_status":"done","queue_duration_s":1.148211,"scheduling_latency_s":1.148232,"redis_calls":13,"redis_duration_s":0.001578,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001035,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010272,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015682,"completed_at":"2026-10-02T16:22:03.643Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000731,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.210362] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.645Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"0a542f802db4ce1aee1d3288","created_at":"2026-10-02T16:22:02.449Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-6cde7b73556548bc","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"35f8df0a677f35ec05201b9c26fb34dd","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.479Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-0a542f802db4ce1aee1d3288: start","job_status":"start","queue_duration_s":1.166225,"scheduling_latency_s":1.166255} gitlab # [ 281.224210] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.667Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"0a542f802db4ce1aee1d3288","created_at":"2026-10-02T16:22:02.449Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-6cde7b73556548bc","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"35f8df0a677f35ec05201b9c26fb34dd","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.479Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-0a542f802db4ce1aee1d3288: done: 0.022161 sec","job_status":"done","queue_duration_s":1.166225,"scheduling_latency_s":1.166255,"redis_calls":13,"redis_duration_s":0.001939,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000663,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001276,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01205,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022161,"completed_at":"2026-10-02T16:22:03.667Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003853,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.265252] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.669Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"4a7177489cf86aafa707a5a6","created_at":"2026-10-02T16:22:02.449Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-6cde7b73556548bc","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"35f8df0a677f35ec05201b9c26fb34dd","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.479Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-4a7177489cf86aafa707a5a6: start","job_status":"start","queue_duration_s":1.190096,"scheduling_latency_s":1.190122} gitlab # [ 281.279132] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.691Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"4a7177489cf86aafa707a5a6","created_at":"2026-10-02T16:22:02.449Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-6cde7b73556548bc","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"35f8df0a677f35ec05201b9c26fb34dd","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:02.479Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-4a7177489cf86aafa707a5a6: done: 0.021785 sec","job_status":"done","queue_duration_s":1.190096,"scheduling_latency_s":1.190122,"redis_calls":13,"redis_duration_s":0.002405,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000825,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00158,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014534,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021785,"completed_at":"2026-10-02T16:22:03.691Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000767,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.320346] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.693Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f05b8ab37e96db4a2be6057b","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f05b8ab37e96db4a2be6057b: start","job_status":"start","queue_duration_s":0.369944,"scheduling_latency_s":0.369972} gitlab # [ 281.333182] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.714Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f05b8ab37e96db4a2be6057b","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f05b8ab37e96db4a2be6057b: done: 0.021657 sec","job_status":"done","queue_duration_s":0.369944,"scheduling_latency_s":0.369972,"redis_calls":13,"redis_duration_s":0.002394,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000976,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001418,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014529,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021657,"completed_at":"2026-10-02T16:22:03.714Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.373819] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.716Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9d7c82e33516054a25c567e","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9d7c82e33516054a25c567e: start","job_status":"start","queue_duration_s":0.393371,"scheduling_latency_s":0.393399} gitlab # [ 281.386727] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.735Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9d7c82e33516054a25c567e","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9d7c82e33516054a25c567e: done: 0.018692 sec","job_status":"done","queue_duration_s":0.393371,"scheduling_latency_s":0.393399,"redis_calls":13,"redis_duration_s":0.001954,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000737,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001217,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012439,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018692,"completed_at":"2026-10-02T16:22:03.735Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.428899] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.736Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6638cab860824d9c90a3f8ff","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6638cab860824d9c90a3f8ff: start","job_status":"start","queue_duration_s":0.413399,"scheduling_latency_s":0.413422} gitlab # [ 281.441849] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.752Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6638cab860824d9c90a3f8ff","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6638cab860824d9c90a3f8ff: done: 0.015752 sec","job_status":"done","queue_duration_s":0.413399,"scheduling_latency_s":0.413422,"redis_calls":13,"redis_duration_s":0.001817,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001269,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010287,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015752,"completed_at":"2026-10-02T16:22:03.752Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.482230] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.753Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb94a0cc1ecfda14950a2719","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb94a0cc1ecfda14950a2719: start","job_status":"start","queue_duration_s":0.430437,"scheduling_latency_s":0.430461} gitlab # [ 281.494985] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.768Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb94a0cc1ecfda14950a2719","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb94a0cc1ecfda14950a2719: done: 0.014684 sec","job_status":"done","queue_duration_s":0.430437,"scheduling_latency_s":0.430461,"redis_calls":13,"redis_duration_s":0.0014780000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000546,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000932,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009636,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014684,"completed_at":"2026-10-02T16:22:03.768Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.536160] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.769Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"79c9f31175130cb6d3644756","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-79c9f31175130cb6d3644756: start","job_status":"start","queue_duration_s":0.446398,"scheduling_latency_s":0.446422} gitlab # [ 281.548697] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.784Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"79c9f31175130cb6d3644756","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-79c9f31175130cb6d3644756: done: 0.014652 sec","job_status":"done","queue_duration_s":0.446398,"scheduling_latency_s":0.446422,"redis_calls":13,"redis_duration_s":0.0014559999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000931,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009581,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014652,"completed_at":"2026-10-02T16:22:03.784Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.589720] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.785Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52e7930a9ede658ea90806d4","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52e7930a9ede658ea90806d4: start","job_status":"start","queue_duration_s":0.462359,"scheduling_latency_s":0.462385} gitlab # [ 281.603737] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.800Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52e7930a9ede658ea90806d4","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52e7930a9ede658ea90806d4: done: 0.014633 sec","job_status":"done","queue_duration_s":0.462359,"scheduling_latency_s":0.462385,"redis_calls":13,"redis_duration_s":0.001467,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000928,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009505,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014633,"completed_at":"2026-10-02T16:22:03.799Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.644677] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.801Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b1970ca44430a4a891df9ef","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b1970ca44430a4a891df9ef: start","job_status":"start","queue_duration_s":0.478302,"scheduling_latency_s":0.478328} gitlab # [ 281.657620] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.816Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b1970ca44430a4a891df9ef","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b1970ca44430a4a891df9ef: done: 0.014977 sec","job_status":"done","queue_duration_s":0.478302,"scheduling_latency_s":0.478328,"redis_calls":13,"redis_duration_s":0.001482,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000945,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009752,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014977,"completed_at":"2026-10-02T16:22:03.816Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.698153] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.817Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f3416b7ed958a399289ae543","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f3416b7ed958a399289ae543: start","job_status":"start","queue_duration_s":0.494586,"scheduling_latency_s":0.49461} gitlab # [ 281.710874] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.831Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f3416b7ed958a399289ae543","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f3416b7ed958a399289ae543: done: 0.014265 sec","job_status":"done","queue_duration_s":0.494586,"scheduling_latency_s":0.49461,"redis_calls":13,"redis_duration_s":0.001454,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000936,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009502,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014265,"completed_at":"2026-10-02T16:22:03.831Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.751819] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.833Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"474d3481390d1eb77bd986d0","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-474d3481390d1eb77bd986d0: start","job_status":"start","queue_duration_s":0.509933,"scheduling_latency_s":0.50995} gitlab # [ 281.764959] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.847Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"474d3481390d1eb77bd986d0","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-474d3481390d1eb77bd986d0: done: 0.01414 sec","job_status":"done","queue_duration_s":0.509933,"scheduling_latency_s":0.50995,"redis_calls":13,"redis_duration_s":0.001576,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001029,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009319,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01414,"completed_at":"2026-10-02T16:22:03.847Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.806463] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.848Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"411a67bf67a02512ffb00e08","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-411a67bf67a02512ffb00e08: start","job_status":"start","queue_duration_s":0.52533,"scheduling_latency_s":0.525348} gitlab # [ 281.819393] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.862Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"411a67bf67a02512ffb00e08","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-411a67bf67a02512ffb00e08: done: 0.013739 sec","job_status":"done","queue_duration_s":0.52533,"scheduling_latency_s":0.525348,"redis_calls":13,"redis_duration_s":0.0014550000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000931,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009437,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013739,"completed_at":"2026-10-02T16:22:03.862Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.860344] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.863Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34d453beb7b7b6c9723151d0","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34d453beb7b7b6c9723151d0: start","job_status":"start","queue_duration_s":0.540211,"scheduling_latency_s":0.540228} gitlab # [ 281.873150] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.876Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34d453beb7b7b6c9723151d0","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34d453beb7b7b6c9723151d0: done: 0.013691 sec","job_status":"done","queue_duration_s":0.540211,"scheduling_latency_s":0.540228,"redis_calls":13,"redis_duration_s":0.001436,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009383,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013691,"completed_at":"2026-10-02T16:22:03.876Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.913609] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.878Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d01164792fdc8d2e02d0128","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d01164792fdc8d2e02d0128: start","job_status":"start","queue_duration_s":0.554996,"scheduling_latency_s":0.555012} gitlab # [ 281.926679] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.891Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d01164792fdc8d2e02d0128","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d01164792fdc8d2e02d0128: done: 0.013632 sec","job_status":"done","queue_duration_s":0.554996,"scheduling_latency_s":0.555012,"redis_calls":13,"redis_duration_s":0.0014269999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000919,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009359,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013632,"completed_at":"2026-10-02T16:22:03.891Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.967477] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.892Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a16f9faf0fd52d041005d97","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a16f9faf0fd52d041005d97: start","job_status":"start","queue_duration_s":0.569753,"scheduling_latency_s":0.569771} gitlab # [ 281.980539] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.906Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a16f9faf0fd52d041005d97","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a16f9faf0fd52d041005d97: done: 0.013677 sec","job_status":"done","queue_duration_s":0.569753,"scheduling_latency_s":0.569771,"redis_calls":13,"redis_duration_s":0.001426,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009409,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013677,"completed_at":"2026-10-02T16:22:03.906Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.022457] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.907Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3754adc1f8606ee915867eda","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3754adc1f8606ee915867eda: start","job_status":"start","queue_duration_s":0.584551,"scheduling_latency_s":0.584568} gitlab # [ 282.035462] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.921Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3754adc1f8606ee915867eda","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3754adc1f8606ee915867eda: done: 0.013725 sec","job_status":"done","queue_duration_s":0.584551,"scheduling_latency_s":0.584568,"redis_calls":13,"redis_duration_s":0.0014629999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000949,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009427,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013725,"completed_at":"2026-10-02T16:22:03.921Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.076609] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.922Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70f4c8140e7e05a07a818712","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70f4c8140e7e05a07a818712: start","job_status":"start","queue_duration_s":0.599391,"scheduling_latency_s":0.599409} gitlab # [ 282.089280] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.936Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70f4c8140e7e05a07a818712","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70f4c8140e7e05a07a818712: done: 0.013592 sec","job_status":"done","queue_duration_s":0.599391,"scheduling_latency_s":0.599409,"redis_calls":13,"redis_duration_s":0.00144,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000907,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009325,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013592,"completed_at":"2026-10-02T16:22:03.935Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.129982] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.937Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d850e3ac06f1c975f3681456","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d850e3ac06f1c975f3681456: start","job_status":"start","queue_duration_s":0.614048,"scheduling_latency_s":0.614064} gitlab # [ 282.142843] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.951Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d850e3ac06f1c975f3681456","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d850e3ac06f1c975f3681456: done: 0.014017 sec","job_status":"done","queue_duration_s":0.614048,"scheduling_latency_s":0.614064,"redis_calls":13,"redis_duration_s":0.001677,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001159,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009523,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014017,"completed_at":"2026-10-02T16:22:03.951Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.183922] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.952Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e45eb56f579defd103b88e7f","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e45eb56f579defd103b88e7f: start","job_status":"start","queue_duration_s":0.629212,"scheduling_latency_s":0.629231} gitlab # [ 282.196888] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.965Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e45eb56f579defd103b88e7f","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e45eb56f579defd103b88e7f: done: 0.013532 sec","job_status":"done","queue_duration_s":0.629212,"scheduling_latency_s":0.629231,"redis_calls":13,"redis_duration_s":0.001414,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000499,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000915,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009323,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013532,"completed_at":"2026-10-02T16:22:03.965Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.238484] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.966Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb7d0e91b3f9ce7c9385f5a7","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb7d0e91b3f9ce7c9385f5a7: start","job_status":"start","queue_duration_s":0.643882,"scheduling_latency_s":0.643899} gitlab # [ 282.259215] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.980Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb7d0e91b3f9ce7c9385f5a7","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb7d0e91b3f9ce7c9385f5a7: done: 0.01372 sec","job_status":"done","queue_duration_s":0.643882,"scheduling_latency_s":0.643899,"redis_calls":13,"redis_duration_s":0.001509,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000967,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009397,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01372,"completed_at":"2026-10-02T16:22:03.980Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.320329] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:03.981Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91c99e3f49724a60e116bca6","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91c99e3f49724a60e116bca6: start","job_status":"start","queue_duration_s":0.658707,"scheduling_latency_s":0.658724} gitlab # [ 282.337906] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.019Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91c99e3f49724a60e116bca6","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91c99e3f49724a60e116bca6: done: 0.037858 sec","job_status":"done","queue_duration_s":0.658707,"scheduling_latency_s":0.658724,"redis_calls":13,"redis_duration_s":0.0022389999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00121,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001029,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032597,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.037858,"completed_at":"2026-10-02T16:22:04.019Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.383851] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.020Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"750eff0e82b7ed6589d1a12b","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-750eff0e82b7ed6589d1a12b: start","job_status":"start","queue_duration_s":0.697873,"scheduling_latency_s":0.697891} gitlab # [ 282.396770] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.035Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"750eff0e82b7ed6589d1a12b","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-750eff0e82b7ed6589d1a12b: done: 0.014942 sec","job_status":"done","queue_duration_s":0.697873,"scheduling_latency_s":0.697891,"redis_calls":13,"redis_duration_s":0.001508,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000968,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01027,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014942,"completed_at":"2026-10-02T16:22:04.035Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.439081] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.037Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a1fbbb51fcc274390bfd0c98","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a1fbbb51fcc274390bfd0c98: start","job_status":"start","queue_duration_s":0.71394,"scheduling_latency_s":0.713957} gitlab # [ 282.452087] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.052Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a1fbbb51fcc274390bfd0c98","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a1fbbb51fcc274390bfd0c98: done: 0.015268 sec","job_status":"done","queue_duration_s":0.71394,"scheduling_latency_s":0.713957,"redis_calls":13,"redis_duration_s":0.0017130000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001201,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009517,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015268,"completed_at":"2026-10-02T16:22:04.052Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.492956] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.054Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d6a2ac851f314b5937b5b89","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d6a2ac851f314b5937b5b89: start","job_status":"start","queue_duration_s":0.731108,"scheduling_latency_s":0.731125} gitlab # [ 282.505726] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.069Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d6a2ac851f314b5937b5b89","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d6a2ac851f314b5937b5b89: done: 0.015044 sec","job_status":"done","queue_duration_s":0.731108,"scheduling_latency_s":0.731125,"redis_calls":13,"redis_duration_s":0.0014850000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000582,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000903,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010525,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015044,"completed_at":"2026-10-02T16:22:04.069Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.546952] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.070Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b1811b4c657b90b0fda97aed","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b1811b4c657b90b0fda97aed: start","job_status":"start","queue_duration_s":0.747248,"scheduling_latency_s":0.747264} gitlab # [ 282.559940] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.085Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b1811b4c657b90b0fda97aed","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b1811b4c657b90b0fda97aed: done: 0.01474 sec","job_status":"done","queue_duration_s":0.747248,"scheduling_latency_s":0.747264,"redis_calls":13,"redis_duration_s":0.001519,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000993,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010458,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01474,"completed_at":"2026-10-02T16:22:04.084Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.600965] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.086Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"87b773cc5c096e84dcf0b8d4","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-87b773cc5c096e84dcf0b8d4: start","job_status":"start","queue_duration_s":0.763071,"scheduling_latency_s":0.763088} gitlab # [ 282.613956] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.100Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"87b773cc5c096e84dcf0b8d4","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-87b773cc5c096e84dcf0b8d4: done: 0.014701 sec","job_status":"done","queue_duration_s":0.763071,"scheduling_latency_s":0.763088,"redis_calls":13,"redis_duration_s":0.0014589999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000917,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010392,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014701,"completed_at":"2026-10-02T16:22:04.100Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.655695] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.101Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d27dd18ca540b260dfd9bf47","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d27dd18ca540b260dfd9bf47: start","job_status":"start","queue_duration_s":0.778853,"scheduling_latency_s":0.77887} gitlab # [ 282.668628] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.118Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d27dd18ca540b260dfd9bf47","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d27dd18ca540b260dfd9bf47: done: 0.016406 sec","job_status":"done","queue_duration_s":0.778853,"scheduling_latency_s":0.77887,"redis_calls":13,"redis_duration_s":0.001791,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000741,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00105,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010292,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016406,"completed_at":"2026-10-02T16:22:04.118Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.709014] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.120Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ad1dcb72cbc65a27f291803","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ad1dcb72cbc65a27f291803: start","job_status":"start","queue_duration_s":0.797572,"scheduling_latency_s":0.797589} gitlab # [ 282.721716] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.136Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ad1dcb72cbc65a27f291803","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ad1dcb72cbc65a27f291803: done: 0.015619 sec","job_status":"done","queue_duration_s":0.797572,"scheduling_latency_s":0.797589,"redis_calls":13,"redis_duration_s":0.001706,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001192,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010892,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015619,"completed_at":"2026-10-02T16:22:04.136Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.761810] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.137Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc7e192f8ebf130a8af743d3","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc7e192f8ebf130a8af743d3: start","job_status":"start","queue_duration_s":0.814391,"scheduling_latency_s":0.814409} gitlab # [ 282.774580] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.152Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc7e192f8ebf130a8af743d3","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc7e192f8ebf130a8af743d3: done: 0.01533 sec","job_status":"done","queue_duration_s":0.814391,"scheduling_latency_s":0.814409,"redis_calls":13,"redis_duration_s":0.001722,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001187,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010632,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01533,"completed_at":"2026-10-02T16:22:04.152Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.815697] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.153Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4164cc88735ff349fea993b","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4164cc88735ff349fea993b: start","job_status":"start","queue_duration_s":0.830789,"scheduling_latency_s":0.830807} gitlab # [ 282.829954] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.168Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4164cc88735ff349fea993b","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4164cc88735ff349fea993b: done: 0.014526 sec","job_status":"done","queue_duration_s":0.830789,"scheduling_latency_s":0.830807,"redis_calls":13,"redis_duration_s":0.001464,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000948,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0093,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014526,"completed_at":"2026-10-02T16:22:04.168Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.871111] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.170Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4debe457f5d3dd7c807c2358","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4debe457f5d3dd7c807c2358: start","job_status":"start","queue_duration_s":0.847405,"scheduling_latency_s":0.847421} gitlab # [ 282.883814] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.185Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4debe457f5d3dd7c807c2358","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4debe457f5d3dd7c807c2358: done: 0.014682 sec","job_status":"done","queue_duration_s":0.847405,"scheduling_latency_s":0.847421,"redis_calls":13,"redis_duration_s":0.001482,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000928,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010394,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014682,"completed_at":"2026-10-02T16:22:04.185Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.924867] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.186Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea1b767a6d224bbf9310e850","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea1b767a6d224bbf9310e850: start","job_status":"start","queue_duration_s":0.863157,"scheduling_latency_s":0.863175} gitlab # [ 282.937620] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.201Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea1b767a6d224bbf9310e850","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea1b767a6d224bbf9310e850: done: 0.014842 sec","job_status":"done","queue_duration_s":0.863157,"scheduling_latency_s":0.863175,"redis_calls":13,"redis_duration_s":0.001508,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000557,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000951,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010517,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014842,"completed_at":"2026-10-02T16:22:04.201Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.977996] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.202Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f1ebbf367975252dff11895","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f1ebbf367975252dff11895: start","job_status":"start","queue_duration_s":0.879073,"scheduling_latency_s":0.87909} gitlab # [ 282.990807] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.218Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f1ebbf367975252dff11895","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f1ebbf367975252dff11895: done: 0.015899 sec","job_status":"done","queue_duration_s":0.879073,"scheduling_latency_s":0.87909,"redis_calls":13,"redis_duration_s":0.001467,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000942,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009278,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015899,"completed_at":"2026-10-02T16:22:04.217Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.032879] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.220Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"488f5849267ef6876107cee6","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-488f5849267ef6876107cee6: start","job_status":"start","queue_duration_s":0.89725,"scheduling_latency_s":0.897267} gitlab # [ 283.046084] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.235Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"488f5849267ef6876107cee6","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-488f5849267ef6876107cee6: done: 0.014706 sec","job_status":"done","queue_duration_s":0.89725,"scheduling_latency_s":0.897267,"redis_calls":13,"redis_duration_s":0.001455,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000917,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010424,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014706,"completed_at":"2026-10-02T16:22:04.234Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.086919] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.236Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8d637c3768815bd928d9a40","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8d637c3768815bd928d9a40: start","job_status":"start","queue_duration_s":0.913049,"scheduling_latency_s":0.913066} gitlab # [ 283.100428] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.251Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8d637c3768815bd928d9a40","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8d637c3768815bd928d9a40: done: 0.015314 sec","job_status":"done","queue_duration_s":0.913049,"scheduling_latency_s":0.913066,"redis_calls":13,"redis_duration_s":0.0017490000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000788,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000961,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010856,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015314,"completed_at":"2026-10-02T16:22:04.251Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.141453] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.252Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fab66952ea506d94de903299","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fab66952ea506d94de903299: start","job_status":"start","queue_duration_s":0.929451,"scheduling_latency_s":0.929467} gitlab # [ 283.154253] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.267Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fab66952ea506d94de903299","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fab66952ea506d94de903299: done: 0.014761 sec","job_status":"done","queue_duration_s":0.929451,"scheduling_latency_s":0.929467,"redis_calls":13,"redis_duration_s":0.001448,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000932,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00936,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014761,"completed_at":"2026-10-02T16:22:04.267Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.194691] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.269Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ea68fd39f9194684c07074a","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ea68fd39f9194684c07074a: start","job_status":"start","queue_duration_s":0.946493,"scheduling_latency_s":0.946511} gitlab # [ 283.207580] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.284Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ea68fd39f9194684c07074a","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ea68fd39f9194684c07074a: done: 0.015398 sec","job_status":"done","queue_duration_s":0.946493,"scheduling_latency_s":0.946511,"redis_calls":13,"redis_duration_s":0.0015439999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000578,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000966,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010902,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015398,"completed_at":"2026-10-02T16:22:04.284Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.249751] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.286Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2913ba6533f8ff80b2ff82ac","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2913ba6533f8ff80b2ff82ac: start","job_status":"start","queue_duration_s":0.962978,"scheduling_latency_s":0.962995} gitlab # [ 283.262780] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.300Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2913ba6533f8ff80b2ff82ac","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2913ba6533f8ff80b2ff82ac: done: 0.014735 sec","job_status":"done","queue_duration_s":0.962978,"scheduling_latency_s":0.962995,"redis_calls":13,"redis_duration_s":0.0015300000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000559,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000971,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010466,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014735,"completed_at":"2026-10-02T16:22:04.300Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.303399] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.301Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"076b719d97c78e63720cff67","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-076b719d97c78e63720cff67: start","job_status":"start","queue_duration_s":0.978807,"scheduling_latency_s":0.978824} gitlab # [ 283.316418] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.316Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"076b719d97c78e63720cff67","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-076b719d97c78e63720cff67: done: 0.014533 sec","job_status":"done","queue_duration_s":0.978807,"scheduling_latency_s":0.978824,"redis_calls":13,"redis_duration_s":0.001471,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000953,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009327,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014533,"completed_at":"2026-10-02T16:22:04.316Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.357194] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.318Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e967fe9ea598bf6a4956ae1","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e967fe9ea598bf6a4956ae1: start","job_status":"start","queue_duration_s":0.995458,"scheduling_latency_s":0.995476} gitlab # [ 283.369918] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.333Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e967fe9ea598bf6a4956ae1","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e967fe9ea598bf6a4956ae1: done: 0.015051 sec","job_status":"done","queue_duration_s":0.995458,"scheduling_latency_s":0.995476,"redis_calls":13,"redis_duration_s":0.001465,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000551,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000914,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010472,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015051,"completed_at":"2026-10-02T16:22:04.333Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.410058] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.334Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9cdcf6d737a1a06c308339dc","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9cdcf6d737a1a06c308339dc: start","job_status":"start","queue_duration_s":1.011717,"scheduling_latency_s":1.011734} gitlab # [ 283.422922] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.350Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9cdcf6d737a1a06c308339dc","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9cdcf6d737a1a06c308339dc: done: 0.015275 sec","job_status":"done","queue_duration_s":1.011717,"scheduling_latency_s":1.011734,"redis_calls":13,"redis_duration_s":0.001533,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001006,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010566,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015275,"completed_at":"2026-10-02T16:22:04.349Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.465173] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.351Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33771896d1824fa849cfc1f2","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33771896d1824fa849cfc1f2: start","job_status":"start","queue_duration_s":1.028082,"scheduling_latency_s":1.028099} gitlab # [ 283.478059] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.366Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33771896d1824fa849cfc1f2","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33771896d1824fa849cfc1f2: done: 0.014962 sec","job_status":"done","queue_duration_s":1.028082,"scheduling_latency_s":1.028099,"redis_calls":13,"redis_duration_s":0.0014889999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000971,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010584,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014962,"completed_at":"2026-10-02T16:22:04.366Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.518571] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.367Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b88fc7fef9a2153bca706365","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b88fc7fef9a2153bca706365: start","job_status":"start","queue_duration_s":1.044186,"scheduling_latency_s":1.044204} gitlab # [ 283.531651] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.383Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b88fc7fef9a2153bca706365","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b88fc7fef9a2153bca706365: done: 0.016466 sec","job_status":"done","queue_duration_s":1.044186,"scheduling_latency_s":1.044204,"redis_calls":13,"redis_duration_s":0.001486,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009618,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016466,"completed_at":"2026-10-02T16:22:04.383Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.572955] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.385Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d4868dfe3e1f91180005de0","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d4868dfe3e1f91180005de0: start","job_status":"start","queue_duration_s":1.06198,"scheduling_latency_s":1.061997} gitlab # [ 283.585662] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.400Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d4868dfe3e1f91180005de0","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d4868dfe3e1f91180005de0: done: 0.015195 sec","job_status":"done","queue_duration_s":1.06198,"scheduling_latency_s":1.061997,"redis_calls":13,"redis_duration_s":0.00147,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000566,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000904,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010741,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015195,"completed_at":"2026-10-02T16:22:04.400Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.626312] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.401Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"529c2a9ccd8bf9aebbc1782b","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-529c2a9ccd8bf9aebbc1782b: start","job_status":"start","queue_duration_s":1.07829,"scheduling_latency_s":1.078307} gitlab # [ 283.639346] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.416Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"529c2a9ccd8bf9aebbc1782b","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-529c2a9ccd8bf9aebbc1782b: done: 0.014991 sec","job_status":"done","queue_duration_s":1.07829,"scheduling_latency_s":1.078307,"redis_calls":13,"redis_duration_s":0.002304,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00135,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000954,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010512,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014991,"completed_at":"2026-10-02T16:22:04.416Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.681605] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.417Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7621229e9d51231f96ec8861","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7621229e9d51231f96ec8861: start","job_status":"start","queue_duration_s":1.094426,"scheduling_latency_s":1.094443} gitlab # [ 283.694572] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.432Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7621229e9d51231f96ec8861","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7621229e9d51231f96ec8861: done: 0.01532 sec","job_status":"done","queue_duration_s":1.094426,"scheduling_latency_s":1.094443,"redis_calls":13,"redis_duration_s":0.001491,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000944,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009643,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01532,"completed_at":"2026-10-02T16:22:04.432Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.735173] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.435Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"18bce61c2e42def1e804ce5c","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-18bce61c2e42def1e804ce5c: start","job_status":"start","queue_duration_s":1.112141,"scheduling_latency_s":1.112159} gitlab # [ 283.748235] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.451Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"18bce61c2e42def1e804ce5c","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-18bce61c2e42def1e804ce5c: done: 0.016246 sec","job_status":"done","queue_duration_s":1.112141,"scheduling_latency_s":1.112159,"redis_calls":13,"redis_duration_s":0.001552,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000618,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000934,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011219,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016246,"completed_at":"2026-10-02T16:22:04.451Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.788986] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.452Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"460cf4ca968d9ac5381b015a","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-460cf4ca968d9ac5381b015a: start","job_status":"start","queue_duration_s":1.129553,"scheduling_latency_s":1.12957} gitlab # [ 283.802132] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.467Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"460cf4ca968d9ac5381b015a","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-460cf4ca968d9ac5381b015a: done: 0.015104 sec","job_status":"done","queue_duration_s":1.129553,"scheduling_latency_s":1.12957,"redis_calls":13,"redis_duration_s":0.001438,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000937,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010564,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015104,"completed_at":"2026-10-02T16:22:04.467Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.843456] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.468Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12abbdebc45eed32be1d0e69","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12abbdebc45eed32be1d0e69: start","job_status":"start","queue_duration_s":1.145833,"scheduling_latency_s":1.145849} gitlab # [ 283.857676] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.484Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12abbdebc45eed32be1d0e69","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12abbdebc45eed32be1d0e69: done: 0.015138 sec","job_status":"done","queue_duration_s":1.145833,"scheduling_latency_s":1.145849,"redis_calls":13,"redis_duration_s":0.0014689999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000953,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010616,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015138,"completed_at":"2026-10-02T16:22:04.483Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.899171] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.485Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a226f623598b765a9b303119","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a226f623598b765a9b303119: start","job_status":"start","queue_duration_s":1.162171,"scheduling_latency_s":1.162188} gitlab # [ 283.912127] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.501Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a226f623598b765a9b303119","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a226f623598b765a9b303119: done: 0.016604 sec","job_status":"done","queue_duration_s":1.162171,"scheduling_latency_s":1.162188,"redis_calls":13,"redis_duration_s":0.0014650000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000937,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010373,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016604,"completed_at":"2026-10-02T16:22:04.501Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.953006] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.503Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28f773669c47c5f805fa686c","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28f773669c47c5f805fa686c: start","job_status":"start","queue_duration_s":1.180056,"scheduling_latency_s":1.180075} gitlab # [ 283.965657] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.518Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28f773669c47c5f805fa686c","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28f773669c47c5f805fa686c: done: 0.015801 sec","job_status":"done","queue_duration_s":1.180056,"scheduling_latency_s":1.180075,"redis_calls":13,"redis_duration_s":0.00155,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011077,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015801,"completed_at":"2026-10-02T16:22:04.518Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.006662] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.520Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66151e7603ca971bbe1ff7be","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66151e7603ca971bbe1ff7be: start","job_status":"start","queue_duration_s":1.197134,"scheduling_latency_s":1.197152} gitlab # [ 284.019499] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.536Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66151e7603ca971bbe1ff7be","created_at":"2026-10-02T16:22:03.221Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-c975988fefdf47a6","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2e4cfe632d970798807bf22a4da10688","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.323Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66151e7603ca971bbe1ff7be: done: 0.015819 sec","job_status":"done","queue_duration_s":1.197134,"scheduling_latency_s":1.197152,"redis_calls":13,"redis_duration_s":0.00151,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00097,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011092,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015819,"completed_at":"2026-10-02T16:22:04.535Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.062157] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.537Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"623cc2982088446760c63ea2","created_at":"2026-10-02T16:22:03.350Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-6b6ef8e35cce4e5e","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"63a571c0abb9cfae7a2972fa89756904","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.364Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-623cc2982088446760c63ea2: start","job_status":"start","queue_duration_s":1.173231,"scheduling_latency_s":1.173248} gitlab # [ 284.075356] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.549Z","message":"No existing merge request to be cleaned up."} gitlab # [ 284.077169] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.559Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"623cc2982088446760c63ea2","created_at":"2026-10-02T16:22:03.350Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-6b6ef8e35cce4e5e","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"63a571c0abb9cfae7a2972fa89756904","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.364Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-623cc2982088446760c63ea2: done: 0.021868 sec","job_status":"done","queue_duration_s":1.173231,"scheduling_latency_s":1.173248,"redis_calls":10,"redis_duration_s":0.001565,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000602,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000963,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014922,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021868,"completed_at":"2026-10-02T16:22:04.559Z","load_balancing_strategy":"primary","db_duration_s":0.002147,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.118001] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.560Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fbea9d08c834119066704f68","created_at":"2026-10-02T16:22:03.350Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-6b6ef8e35cce4e5e","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"63a571c0abb9cfae7a2972fa89756904","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.364Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fbea9d08c834119066704f68: start","job_status":"start","queue_duration_s":1.196212,"scheduling_latency_s":1.196229} gitlab # [ 284.131161] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.570Z","message":"No existing merge request to be cleaned up."} gitlab # [ 284.132970] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.579Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fbea9d08c834119066704f68","created_at":"2026-10-02T16:22:03.350Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-6b6ef8e35cce4e5e","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"63a571c0abb9cfae7a2972fa89756904","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.364Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fbea9d08c834119066704f68: done: 0.018679 sec","job_status":"done","queue_duration_s":1.196212,"scheduling_latency_s":1.196229,"redis_calls":10,"redis_duration_s":0.00131,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000763,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013725,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018679,"completed_at":"2026-10-02T16:22:04.578Z","load_balancing_strategy":"primary","db_duration_s":0.001567,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.173579] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.580Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"cea674bd864f6519c1c87f25","created_at":"2026-10-02T16:22:03.350Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-6b6ef8e35cce4e5e","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"63a571c0abb9cfae7a2972fa89756904","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.364Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-cea674bd864f6519c1c87f25: start","job_status":"start","queue_duration_s":1.215963,"scheduling_latency_s":1.21598} gitlab # [ 284.186632] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.590Z","message":"No existing merge request to be cleaned up."} gitlab # [ 284.188454] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.599Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"cea674bd864f6519c1c87f25","created_at":"2026-10-02T16:22:03.350Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-6b6ef8e35cce4e5e","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"63a571c0abb9cfae7a2972fa89756904","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.364Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-cea674bd864f6519c1c87f25: done: 0.019401 sec","job_status":"done","queue_duration_s":1.215963,"scheduling_latency_s":1.21598,"redis_calls":10,"redis_duration_s":0.001363,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000581,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000782,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013878,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019401,"completed_at":"2026-10-02T16:22:04.599Z","load_balancing_strategy":"primary","db_duration_s":0.001466,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.228378] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.600Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"092f6207bc1684abb859a378","created_at":"2026-10-02T16:22:03.350Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-6b6ef8e35cce4e5e","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"63a571c0abb9cfae7a2972fa89756904","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.364Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-092f6207bc1684abb859a378: start","job_status":"start","queue_duration_s":1.236542,"scheduling_latency_s":1.23656} gitlab # [ 284.241372] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.611Z","message":"No existing merge request to be cleaned up."} gitlab # [ 284.243113] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:04.619Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"092f6207bc1684abb859a378","created_at":"2026-10-02T16:22:03.350Z","trace_propagation_headers":{"sentry-trace":"030bb19488044a129feb6a2389ce12a5-6b6ef8e35cce4e5e","baggage":"sentry-trace_id=030bb19488044a129feb6a2389ce12a5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"63a571c0abb9cfae7a2972fa89756904","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:03.364Z","job_size_bytes":2,"pid":1998,"sidekiq_tid":"map2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-092f6207bc1684abb859a378: done: 0.018481 sec","job_status":"done","queue_duration_s":1.236542,"scheduling_latency_s":1.23656,"redis_calls":10,"redis_duration_s":0.0014390000000000002,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000555,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000884,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013443,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018481,"completed_at":"2026-10-02T16:22:04.619Z","load_balancing_strategy":"primary","db_duration_s":0.001434,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 302.934065] postgres[3329]: [3329] ERROR: relation "table_without_models" does not exist gitlab # [ 302.935860] postgres[3329]: [3329] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 308.154470] redis[833]: 10 changes in 300 seconds. Saving... gitlab # [ 308.158091] redis[833]: Background saving started by pid 3368 gitlab # [ 308.161730] redis[3368]: BGSAVE done, 457 keys saved, 0 keys skipped, 118282 bytes written. gitlab # [ 308.165238] redis[3368]: DB saved on disk gitlab # [ 308.166487] redis[3368]: Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB gitlab # [ 308.255377] redis[833]: Background saving terminated with success gitlab # [ 309.050542] runuser[2984]: pam_unix(runuser:session): session closed for user gitlab gitlab # [ 309.053069] setup-gitlab-start[2958]: + systemctl restart gitlab-sidekiq.service gitlab # [ 309.081677] systemd[1]: Stopping gitlab-sidekiq.service... gitlab # [ 309.083763] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:31.931Z","message":"Shutting down"} gitlab # [ 309.086344] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:31.932Z","message":"Scheduler exiting..."} gitlab # [ 309.088751] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:31.932Z","message":"Terminating quiet threads for default capsule"} gitlab # [ 309.091475] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:31.932Z","message":"Scheduler exiting..."} gitlab # [ 309.584116] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:32.433Z","message":"Pausing to allow jobs to finish..."} gitlab # [ 309.778518] postgres[923]: [923] LOG: checkpoint starting: time gitlab # [ 311.068983] gitaly[948]: time="2026-10-02T16:22:33.918Z" level=info msg="burden monitor stats" pid=948 gitlab # [ 312.088334] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:34.937Z","message":"Bye!"} gitlab # [ 312.114591] sidekiq-cluster[1998]: {"severity":"INFO","time":"2026-10-02T16:22:34.964Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1998,"worker_id":"sidekiq_0","memwd_rss_bytes":924459008,"retry":0} gitlab # [ 312.592869] systemd[1]: gitlab-sidekiq.service: Deactivated successfully. gitlab # [ 312.594543] systemd[1]: Stopped gitlab-sidekiq.service. gitlab # [ 312.595785] systemd[1]: gitlab-sidekiq.service: Consumed 1min 49.586s CPU time over 3min 23.563s wall clock time, 942.2M memory peak, 5M read from disk. gitlab # [ 312.600471] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 312.603156] setup-gitlab-start[2958]: + mkdir -p /var/lib/secrets gitlab # [ 312.606276] setup-gitlab-start[2958]: + echo nixbot-test-token-of-20-chars gitlab # [ 312.607830] setup-gitlab-start[2958]: + chmod 644 /var/lib/secrets/gitlab-token gitlab # [ 312.609534] setup-gitlab-start[2958]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars gitlab # [ 313.426246] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:22:36 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 791 gitlab # [ 314.122521] gitaly[948]: time="2026-10-02T16:22:36.972Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSEG28GDCBM1JFKV7V93S error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:22:46.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:22:36.971" pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 314.122836] gitaly[948]: time="2026-10-02T16:22:36.972Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSEG28GDCBM1JFKV7V93S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:22:46.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:22:36.971" grpc.time_ms=0.82 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 314.313526] gitaly[948]: time="2026-10-02T16:22:37.163Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSEG28GDCBM1JFKV7V93S error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:22:47.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:22:37.163" pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 314.330492] gitaly[948]: time="2026-10-02T16:22:37.163Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSEG28GDCBM1JFKV7V93S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:22:47.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:22:37.163" grpc.time_ms=0.437 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 314.360498] gitaly[948]: time="2026-10-02T16:22:37.210Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSEG28GDCBM1JFKV7V93S diskcache=42d34280-99b2-4ffa-a63c-f8721a41924a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-02T16:22:47.012" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:22:37.172" pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 314.377670] gitaly[948]: time="2026-10-02T16:22:37.211Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1441 command.oublock=32 command.real_time_ms=22 command.spawn_token_fork_ms=1 command.system_time_ms=18 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSEG28GDCBM1JFKV7V93S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-02T16:22:47.012" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:22:37.172" grpc.time_ms=38.197 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 314.397929] gitaly[948]: time="2026-10-02T16:22:37.224Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSEG28GDCBM1JFKV7V93S grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:22:47.063" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:22:37.223" pid=948 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 314.411912] gitaly[948]: time="2026-10-02T16:22:37.224Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSEG28GDCBM1JFKV7V93S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:22:47.063" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:22:37.223" grpc.time_ms=0.799 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 314.425992] gitaly[948]: time="2026-10-02T16:22:37.239Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSEG28GDCBM1JFKV7V93S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T16:22:47.068" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:22:37.229" grpc.time_ms=9.876 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 314.470819] gitaly[948]: time="2026-10-02T16:22:37.321Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSEG28GDCBM1JFKV7V93S error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:22:47.160" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:22:37.320" pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 314.487757] gitaly[948]: time="2026-10-02T16:22:37.321Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSEG28GDCBM1JFKV7V93S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:22:47.160" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:22:37.320" grpc.time_ms=0.581 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 314.506382] gitaly[948]: time="2026-10-02T16:22:37.355Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSEG28GDCBM1JFKV7V93S diskcache=0e64b627-f662-4feb-9f7c-bf83e57e0b65 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-02T16:22:47.163" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:22:37.323" pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 314.521975] gitaly[948]: time="2026-10-02T16:22:37.356Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1437 command.oublock=32 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=11 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSEG28GDCBM1JFKV7V93S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-02T16:22:47.163" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:22:37.323" grpc.time_ms=32.544 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 314.540261] gitaly[948]: time="2026-10-02T16:22:37.362Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSEG28GDCBM1JFKV7V93S grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:22:47.201" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:22:37.362" pid=948 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 314.553603] gitaly[948]: time="2026-10-02T16:22:37.363Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSEG28GDCBM1JFKV7V93S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:22:47.201" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:22:37.362" grpc.time_ms=0.989 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 314.566447] gitaly[948]: time="2026-10-02T16:22:37.375Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSEG28GDCBM1JFKV7V93S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T16:22:47.206" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:22:37.366" grpc.time_ms=8.18 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 314.581738] gitaly[948]: time="2026-10-02T16:22:37.409Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSEG28GDCBM1JFKV7V93S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:22:47.248" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:22:37.408" grpc.time_ms=0.211 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 314.593443] gitaly[948]: time="2026-10-02T16:22:37.416Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSEG28GDCBM1JFKV7V93S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-02T16:22:47.255" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:22:37.416" grpc.time_ms=0.468 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 315.128789] gitaly[948]: time="2026-10-02T16:22:37.978Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSEG28GDCBM1JFKV7V93S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T16:22:47.006" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:22:37.967" grpc.time_ms=10.834 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 315.544498] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:22:38 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 2103 gitlab # [ 315.547735] setup-gitlab-start[3406]: {"id":1,"message":null,"error":null} gitlab # [ 317.761678] sidekiq-cluster[3390]: {"severity":"INFO","time":"2026-10-02T16:22:40.582Z","message":"Starting cluster with 1 processes"} gitlab # [ 321.128238] gitaly[948]: time="2026-10-02T16:22:43.977Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSNEVRJH03SENN71S1W0T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T16:22:53.006" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:22:43.967" grpc.time_ms=10.396 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 321.705548] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:22:44 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 1133 gitlab # [ 321.710807] setup-gitlab-start[2958]: + mkdir -p /tmp/test-flake gitlab # [ 321.713733] setup-gitlab-start[2958]: + cd /tmp/test-flake gitlab # [ 321.715188] setup-gitlab-start[2958]: + git init -b master gitlab # [ 321.737066] setup-gitlab-start[3491]: Initialized empty Git repository in /tmp/test-flake/.git/ gitlab # [ 321.739112] setup-gitlab-start[2958]: + git config user.name test gitlab # [ 321.741471] setup-gitlab-start[2958]: + git config user.email test@example.com gitlab # [ 321.745332] setup-gitlab-start[2958]: + cat gitlab # [ 321.748387] setup-gitlab-start[2958]: + git add flake.nix gitlab # [ 321.753122] setup-gitlab-start[2958]: + git commit -m 'initial commit' gitlab # [ 321.776890] setup-gitlab-start[3496]: [master (root-commit) e804315] initial commit gitlab # [ 321.778629] setup-gitlab-start[3496]: 1 file changed, 10 insertions(+) gitlab # [ 321.780063] setup-gitlab-start[3496]: create mode 100644 flake.nix gitlab # [ 321.781565] setup-gitlab-start[2958]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git gitlab # [ 321.784279] setup-gitlab-start[2958]: + git push -u origin master gitlab # [ 321.864925] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:22:44 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 52 gitlab # [ 322.145508] gitaly[948]: time="2026-10-02T16:22:44.995Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPSPQC6Z4X1WRCQ4Q1DMJH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:22:44.983" grpc.time_ms=11.585 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 322.167843] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:22:44 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 279 gitlab # [ 322.449622] gitaly[948]: time="2026-10-02T16:22:45.299Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T16:22:55.128" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:22:45.289" grpc.time_ms=10.09 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 322.524717] gitaly[948]: time="2026-10-02T16:22:45.374Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=302 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-10-02T16:23:15.063" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-10-02T16:22:45.364" grpc.time_ms=10.431 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 322.556245] gitaly[948]: time="2026-10-02T16:22:45.406Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 duration_ms=214ns method=POST pid=948 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 322.603384] gitaly[948]: time="2026-10-02T16:22:45.453Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 duration_ms=45ns method=POST pid=948 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 322.608519] gitaly[948]: time="2026-10-02T16:22:45.453Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:22:45.190" grpc.time_ms=262.952 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 322.642158] gitaly[948]: time="2026-10-02T16:22:45.492Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:22:45.491" grpc.time_ms=0.932 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 322.692571] gitaly[948]: time="2026-10-02T16:22:45.542Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:22:45.541" grpc.time_ms=0.758 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 322.740459] gitaly[948]: time="2026-10-02T16:22:45.590Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:22:45.589" grpc.time_ms=1.112 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 322.868922] gitaly[948]: time="2026-10-02T16:22:45.719Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T16:22:55.008" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:22:45.708" grpc.time_ms=10.249 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 322.920868] gitaly[948]: time="2026-10-02T16:22:45.771Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1433 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-10-02T16:22:55.046" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:22:45.746" grpc.time_ms=24.138 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 322.943059] gitaly[948]: time="2026-10-02T16:22:45.784Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 duration_ms=155ns method=POST pid=948 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 322.948359] gitaly[948]: time="2026-10-02T16:22:45.785Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:22:45.628" grpc.time_ms=157.071 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 322.965001] gitaly[948]: time="2026-10-02T16:22:45.796Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 diskcache=5368b331-685f-4ce1-8dbb-c4c7c6cc7995 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:22:45.120" pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 322.979437] systemd[1]: Finished setup-gitlab.service. gitlab # [ 322.980864] gitlab-workhorse[885]: time="2026-10-02T16:22:45Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 322.983195] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:22:45 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 790 gitlab # [ 322.985828] setup-gitlab-start[3500]: To http://gitlab/root/test-flake.git gitlab # [ 322.986924] setup-gitlab-start[3500]: * [new branch] master -> master gitlab # [ 322.988010] setup-gitlab-start[3500]: branch 'master' set up to track 'origin/master'. gitlab # [ 322.989283] gitaly[948]: time="2026-10-02T16:22:45.797Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=216 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=851968 command.minflt=13073 command.oublock=32 command.real_time_ms=670 command.spawn_token_fork_ms=0 command.system_time_ms=119 command.user_time_ms=97 component=gitaly.StreamServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:22:45.120" grpc.time_ms=676.518 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 323.004870] systemd[1]: Starting nixbot CI service... gitlab: (finished: waiting for unit setup-gitlab.service, in 324.05 seconds) gitlab: waiting for unit nixbot.service gitlab # [ 324.190311] nixbot[3579]: {"ts": "2026-10-02T16:22:47.039Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} gitlab # [ 324.224337] nixbot[3579]: {"ts": "2026-10-02T16:22:47.074Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} gitlab # [ 324.248046] nixbot[3579]: {"ts": "2026-10-02T16:22:47.098Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} gitlab # [ 324.253044] nixbot[3579]: {"ts": "2026-10-02T16:22:47.103Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} gitlab # [ 324.257934] nixbot[3579]: {"ts": "2026-10-02T16:22:47.108Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} gitlab # [ 324.264340] nixbot[3579]: {"ts": "2026-10-02T16:22:47.114Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} gitlab # [ 324.268888] nixbot[3579]: {"ts": "2026-10-02T16:22:47.119Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} gitlab # [ 324.272449] nixbot[3579]: {"ts": "2026-10-02T16:22:47.121Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} gitlab # [ 324.276887] nixbot[3579]: {"ts": "2026-10-02T16:22:47.127Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} gitlab # [ 324.280336] nixbot[3579]: {"ts": "2026-10-02T16:22:47.128Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} gitlab # [ 324.283834] nixbot[3579]: {"ts": "2026-10-02T16:22:47.133Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} gitlab # [ 324.289810] nixbot[3579]: {"ts": "2026-10-02T16:22:47.139Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} gitlab # [ 324.293527] nixbot[3579]: {"ts": "2026-10-02T16:22:47.143Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} gitlab # [ 324.297085] nixbot[3579]: {"ts": "2026-10-02T16:22:47.145Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} gitlab # [ 324.300717] nixbot[3579]: {"ts": "2026-10-02T16:22:47.149Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} gitlab # [ 324.304315] nixbot[3579]: {"ts": "2026-10-02T16:22:47.150Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} gitlab # [ 324.307655] nixbot[3579]: {"ts": "2026-10-02T16:22:47.152Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} gitlab # [ 324.311344] nixbot[3579]: {"ts": "2026-10-02T16:22:47.153Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} gitlab # [ 324.314841] nixbot[3579]: {"ts": "2026-10-02T16:22:47.157Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} gitlab # [ 324.318426] nixbot[3579]: {"ts": "2026-10-02T16:22:47.159Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"} gitlab # [ 324.322015] nixbot[3579]: {"ts": "2026-10-02T16:22:47.160Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"} gitlab # [ 324.325520] nixbot[3579]: {"ts": "2026-10-02T16:22:47.165Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"} gitlab # [ 324.328950] nixbot[3579]: {"ts": "2026-10-02T16:22:47.167Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"} gitlab # [ 324.332176] nixbot[3579]: {"ts": "2026-10-02T16:22:47.168Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"} gitlab # [ 324.335207] nixbot[3579]: {"ts": "2026-10-02T16:22:47.169Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"} gitlab # [ 324.338390] nixbot[3579]: {"ts": "2026-10-02T16:22:47.172Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"} gitlab # [ 324.341527] nixbot[3579]: {"ts": "2026-10-02T16:22:47.173Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"} gitlab # [ 324.344546] nixbot[3579]: {"ts": "2026-10-02T16:22:47.174Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"} gitlab # [ 324.347481] nixbot[3579]: {"ts": "2026-10-02T16:22:47.175Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"} gitlab # [ 324.350616] nixbot[3579]: {"ts": "2026-10-02T16:22:47.188Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"} gitlab # [ 324.353528] nixbot[3579]: {"ts": "2026-10-02T16:22:47.193Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"} gitlab # [ 324.356268] nixbot[3579]: {"ts": "2026-10-02T16:22:47.195Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"} gitlab # [ 324.359104] nixbot[3579]: {"ts": "2026-10-02T16:22:47.196Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"} gitlab # [ 324.361936] nixbot[3579]: {"ts": "2026-10-02T16:22:47.199Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"} gitlab # [ 324.364839] nixbot[3579]: {"ts": "2026-10-02T16:22:47.200Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"} gitlab # [ 324.367480] nixbot[3579]: {"ts": "2026-10-02T16:22:47.206Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"} gitlab # [ 324.370370] nixbot[3579]: {"ts": "2026-10-02T16:22:47.208Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 36, "migration": "pr_approvals"} gitlab # [ 324.713168] nixbot[3579]: INFO: Started server process [3579] gitlab # [ 324.714689] nixbot[3579]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit) gitlab # [ 324.735555] nixbot[3579]: INFO: ::ffff:127.0.0.1:41814 - "GET /health HTTP/1.1" 200 OK gitlab # [ 324.743193] systemd[1]: Started nixbot CI service. gitlab # [ 324.744640] systemd[1]: Reached target Multi-User System. gitlab # [ 324.746506] systemd[1]: Startup finished in 757ms (kernel) + 3.747s (initrd) + 5min 20.242s (userspace) = 5min 24.746s. gitlab # [ 325.284728] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:22:48 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 544 gitlab # [ 325.290061] nixbot[3579]: {"ts": "2026-10-02T16:22:48.131Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 325.315689] nixbot[3579]: {"ts": "2026-10-02T16:22:48.165Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} gitlab # [ 325.405151] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:22:48 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 81 gitlab # [ 325.407354] nixbot[3579]: {"ts": "2026-10-02T16:22:48.256Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 325.409855] nixbot[3579]: {"ts": "2026-10-02T16:22:48.258Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} gitlab # [ 325.581329] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:22:48 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 918 "" "python-httpx/0.28.1" 169 gitlab # [ 325.583597] nixbot[3579]: {"ts": "2026-10-02T16:22:48.432Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""} gitlab: (finished: waiting for unit nixbot.service, in 2.08 seconds) gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health gitlab # [ 325.705321] nixbot[3579]: INFO: ::ffff:127.0.0.1:41824 - "GET /health HTTP/1.1" 200 OK gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.03 seconds) subtest: project discovered and webhook registered gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 325.800850] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:22:48 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "curl/8.21.0" 67 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.10 seconds) (finished: subtest: project discovered and webhook registered, in 0.10 seconds) subtest: push delivers the webhook gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails' gitlab # [ 326.011247] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:22:48 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 420 gitlab # [ 326.015267] nixbot[3579]: {"ts": "2026-10-02T16:22:48.862Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 326.148108] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:22:48 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "python-httpx/0.28.1" 123 gitlab # [ 326.151616] nixbot[3579]: {"ts": "2026-10-02T16:22:48.995Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 326.241598] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:22:49 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 918 "" "python-httpx/0.28.1" 83 gitlab # [ 326.247560] nixbot[3579]: {"ts": "2026-10-02T16:22:49.093Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""} gitlab # [ 326.352948] gitaly[948]: time="2026-10-02T16:22:49.202Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSV0MYGH3WH3M4X1V4PXB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-10-02T16:23:19.085" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:22:49.185" grpc.time_ms=16.006 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 326.377191] gitaly[948]: time="2026-10-02T16:22:49.223Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSV0MYGH3WH3M4X1V4PXB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:23:19.121" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:22:49.222" grpc.time_ms=1.04 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 326.394115] gitaly[948]: time="2026-10-02T16:22:49.241Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSV0MYGH3WH3M4X1V4PXB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-10-02T16:22:59.067" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:22:49.228" grpc.time_ms=13.221 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 326.430796] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:22:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 172 gitlab # [ 326.434840] nixbot[3579]: {"ts": "2026-10-02T16:22:49.282Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""} gitlab # [ 326.756446] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:22:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 319 gitlab # [ 326.759155] nixbot[3579]: {"ts": "2026-10-02T16:22:49.608Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 326.764685] nixbot[3579]: {"ts": "2026-10-02T16:22:49.614Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} gitlab # [ 326.998002] gitaly[948]: time="2026-10-02T16:22:49.848Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YPSVHR4KVQN76G4J3BKRVR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:22:49.835" grpc.time_ms=12.935 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 327.011995] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:22:49 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 191 gitlab # [ 327.156426] gitaly[948]: time="2026-10-02T16:22:50.006Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSVQTQ7M2RR1STWYP72Z8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:22:49.995" pid=948 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc gitlab # [ 327.173435] gitaly[948]: time="2026-10-02T16:22:50.006Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSVQTQ7M2RR1STWYP72Z8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:22:49.995" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 327.195534] gitlab-workhorse[885]: time="2026-10-02T16:22:50Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=113 gitlab # [ 327.199051] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:22:50 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 157 gitlab # [ 327.202096] gitaly[948]: time="2026-10-02T16:22:50.007Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSVQTQ7M2RR1STWYP72Z8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:22:49.995" grpc.time_ms=11.339 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 327.232857] gitaly[948]: time="2026-10-02T16:22:50.083Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSVWVNGE3MZRY4258E1J0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:22:50.066" pid=948 remote_ip="2001:db8:1::1" request_sha=7eb65d0bc6efd02f55443241249f240376fa3f79 response_bytes=223 span.kind=server system=grpc gitlab # [ 327.247360] gitaly[948]: time="2026-10-02T16:22:50.083Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1137 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSVWVNGE3MZRY4258E1J0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:22:50.066" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 327.267052] gitlab-workhorse[885]: time="2026-10-02T16:22:50Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=223 gitlab # [ 327.270092] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:22:50 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 71 gitlab # [ 327.272634] gitaly[948]: time="2026-10-02T16:22:50.083Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1137 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSVWVNGE3MZRY4258E1J0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:22:50.066" grpc.time_ms=16.963 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 327.482854] gitaly[948]: time="2026-10-02T16:22:50.332Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPSW15KDVJS69JF3XNN0RT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:22:50.321" grpc.time_ms=10.354 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 327.504806] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:22:50 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 184 gitlab # [ 327.567347] gitaly[948]: time="2026-10-02T16:22:50.417Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSW72TF79P53D3JVM3BR9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:22:50.406" pid=948 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 327.584407] gitaly[948]: time="2026-10-02T16:22:50.417Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSW72TF79P53D3JVM3BR9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:22:50.406" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 327.606193] gitlab-workhorse[885]: time="2026-10-02T16:22:50Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 327.609143] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:22:50 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 79 gitlab # [ 327.611871] gitaly[948]: time="2026-10-02T16:22:50.417Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSW72TF79P53D3JVM3BR9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:22:50.406" grpc.time_ms=11.639 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 327.814206] gitaly[948]: time="2026-10-02T16:22:50.664Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3YPSWBFDZ3Y4HVP3360EQDN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:22:50.653" grpc.time_ms=10.752 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 327.828165] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:22:50 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 184 gitlab # [ 327.919707] gitaly[948]: time="2026-10-02T16:22:50.769Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSWHBDXH8HK47TNMBPZJP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:22:50.750" pid=948 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc gitlab # [ 327.938014] gitaly[948]: time="2026-10-02T16:22:50.770Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1102 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSWHBDXH8HK47TNMBPZJP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:22:50.750" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 327.958155] gitlab-workhorse[885]: time="2026-10-02T16:22:50Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=209 gitlab # [ 327.960678] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:22:50 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 103 gitlab # [ 327.963396] gitaly[948]: time="2026-10-02T16:22:50.770Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1102 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSWHBDXH8HK47TNMBPZJP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:22:50.750" grpc.time_ms=19.717 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 328.182249] gitaly[948]: time="2026-10-02T16:22:51.032Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSWP0JVJQE3Z8R2DBBT3Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:23:21.030" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:22:51.031" grpc.time_ms=0.853 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 328.249195] gitaly[948]: time="2026-10-02T16:22:51.099Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=588 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPSWP0JVJQE3Z8R2DBBT3Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-10-02T16:23:21.087" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:22:51.087" grpc.time_ms=11.452 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 328.301843] gitaly[948]: time="2026-10-02T16:22:51.152Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M3YPSWP0JVJQE3Z8R2DBBT3Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:23:01.000" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:22:51.141" grpc.time_ms=10.574 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 328.333361] gitaly[948]: time="2026-10-02T16:22:51.183Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPSWP0JVJQE3Z8R2DBBT3Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:23:01.032" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:22:51.173" grpc.time_ms=10.1 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 328.514025] gitaly[948]: time="2026-10-02T16:22:51.363Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSWP0JVJQE3Z8R2DBBT3Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-02T16:23:01.222" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:22:51.363" grpc.time_ms=0.479 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 328.552467] gitaly[948]: time="2026-10-02T16:22:51.402Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSWP0JVJQE3Z8R2DBBT3Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-02T16:23:01.252" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:22:51.392" grpc.time_ms=9.629 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 328.600752] gitaly[948]: time="2026-10-02T16:22:51.450Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPSWP0JVJQE3Z8R2DBBT3Q grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:22:51.449" grpc.time_ms=1.31 pid=948 span.kind=server system=grpc gitlab # [ 328.645734] gitaly[948]: time="2026-10-02T16:22:51.495Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPSWP0JVJQE3Z8R2DBBT3Q grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:22:51.495" grpc.time_ms=0.58 pid=948 span.kind=server system=grpc gitlab # [ 328.666899] gitaly[948]: time="2026-10-02T16:22:51.517Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSWP0JVJQE3Z8R2DBBT3Q diskcache=92956fdd-4347-48ee-9743-e14e725e9de2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T16:23:01.268" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:22:51.409" pid=948 ref=refs/keep-around/e8043152c9073e6f344ce0861c23c8506ac05bcf remote_ip="2001:db8:1::1" revision=e8043152c9073e6f344ce0861c23c8506ac05bcf span.kind=server system=grpc username=root gitlab # [ 328.681653] gitaly[948]: time="2026-10-02T16:22:51.517Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=80 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5418 command.oublock=8 command.real_time_ms=101 command.spawn_token_fork_ms=3 command.system_time_ms=18 command.user_time_ms=62 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSWP0JVJQE3Z8R2DBBT3Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T16:23:01.268" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:22:51.409" grpc.time_ms=107.824 pid=948 ref=refs/keep-around/e8043152c9073e6f344ce0861c23c8506ac05bcf remote_ip="2001:db8:1::1" revision=e8043152c9073e6f344ce0861c23c8506ac05bcf span.kind=server system=grpc username=root gitlab # [ 328.698149] gitaly[948]: time="2026-10-02T16:22:51.525Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSWP0JVJQE3Z8R2DBBT3Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:23:21.524" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:22:51.525" grpc.time_ms=0.478 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 329.222582] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:22:52 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e8043152c9073e6f344ce0861c23c8506ac05bcf HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 1255 gitlab # [ 329.231166] nixbot[3579]: {"ts": "2026-10-02T16:22:52.078Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e8043152c9073e6f344ce0861c23c8506ac05bcf \"HTTP/1.1 201 Created\""} gitlab # [ 329.238066] nixbot[3579]: {"ts": "2026-10-02T16:22:52.088Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-e8043152c907-0"} gitlab # [ 329.435873] nixbot[3579]: {"ts": "2026-10-02T16:22:52.285Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-e8043152c907-0", "argv0": "bwrap"} gitlab # [ 329.561113] systemd[1]: Started Nix Daemon. gitlab # [ 329.656878] nix-daemon[3748]: accepted connection from pid 3743, user nixbot gitlab # [ 334.482791] nix-daemon[3748]: accepted connection from pid 3799, user nixbot gitlab # [ 334.499425] nix-daemon[3748]: accepted connection from pid 3743, user nixbot gitlab # [ 338.930652] nix-daemon[3748]: accepted connection from pid 3743, user nixbot gitlab # [ 339.147952] gitaly[948]: time="2026-10-02T16:23:01.997Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPT7ETX9ZW2M32NCHB0DRB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:23:31.695" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:23:01.995" grpc.time_ms=1.001 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 339.256414] gitaly[948]: time="2026-10-02T16:23:02.106Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YPT7ETX9ZW2M32NCHB0DRB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:23:12.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:23:02.094" grpc.time_ms=11.637 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 340.279915] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:23:03 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e8043152c9073e6f344ce0861c23c8506ac05bcf HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 1278 gitlab # [ 340.285079] nixbot[3579]: {"ts": "2026-10-02T16:23:03.131Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e8043152c9073e6f344ce0861c23c8506ac05bcf \"HTTP/1.1 201 Created\""} gitlab # [ 340.382750] gitaly[948]: time="2026-10-02T16:23:03.232Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPT8Q1V8JNRM7WTKQJM733 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:23:33.230" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:23:03.230" grpc.time_ms=1.066 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 340.433506] gitaly[948]: time="2026-10-02T16:23:03.283Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M3YPT8Q1V8JNRM7WTKQJM733 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:23:13.131" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:23:03.271" grpc.time_ms=12.056 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 340.521240] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:23:03 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e8043152c9073e6f344ce0861c23c8506ac05bcf HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 233 gitlab # [ 340.525583] nixbot[3579]: {"ts": "2026-10-02T16:23:03.372Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e8043152c9073e6f344ce0861c23c8506ac05bcf \"HTTP/1.1 201 Created\""} gitlab # [ 340.653436] gitaly[948]: time="2026-10-02T16:23:03.503Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPT8YRBDP5K3KSQBZP7GY3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:23:33.001" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:23:03.502" grpc.time_ms=1.057 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 340.851866] gitaly[948]: time="2026-10-02T16:23:03.702Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YPT8YRBDP5K3KSQBZP7GY3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:23:13.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:23:03.689" grpc.time_ms=12.7 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 341.030189] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:23:03 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e8043152c9073e6f344ce0861c23c8506ac05bcf HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 495 gitlab # [ 341.036494] nixbot[3579]: {"ts": "2026-10-02T16:23:03.886Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e8043152c9073e6f344ce0861c23c8506ac05bcf \"HTTP/1.1 201 Created\""} gitlab # [ 341.525215] gitaly[948]: time="2026-10-02T16:23:04.374Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPT9V6RPB6Q3DG6B34GSAR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:23:04.364" grpc.time_ms=10.356 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 341.547743] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:23:04 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 80 gitlab # [ 341.617482] gitaly[948]: time="2026-10-02T16:23:04.467Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPT9XRPN33FBCE3EG05DKE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:23:04.454" pid=948 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 341.637454] gitaly[948]: time="2026-10-02T16:23:04.467Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPT9XRPN33FBCE3EG05DKE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:23:04.454" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 341.655163] gitlab-workhorse[885]: time="2026-10-02T16:23:04Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 341.657512] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:23:04 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 91 gitlab # [ 341.659715] gitaly[948]: time="2026-10-02T16:23:04.468Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPT9XRPN33FBCE3EG05DKE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:23:04.454" grpc.time_ms=13.09 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 341.755750] gitaly[948]: time="2026-10-02T16:23:04.605Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=577 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPTA29274YEPSNB50A98NZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:23:04.596" grpc.time_ms=8.807 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 341.769631] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:23:04 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 85 gitlab # [ 341.851699] gitaly[948]: time="2026-10-02T16:23:04.701Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPTA51ND90T4SS91B6EP4Y grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:23:04.690" pid=948 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc gitlab # [ 341.870921] gitaly[948]: time="2026-10-02T16:23:04.701Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPTA51ND90T4SS91B6EP4Y grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:23:04.690" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 341.890505] gitlab-workhorse[885]: time="2026-10-02T16:23:04Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 341.893029] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:23:04 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 93 gitlab # [ 341.895379] gitaly[948]: time="2026-10-02T16:23:04.702Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPTA51ND90T4SS91B6EP4Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:23:04.690" grpc.time_ms=12.076 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 360.545089] sidekiq-cluster[3455]: 2026-10-02T16:23:23.394Z pid=3455 tid=yqf INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>3455, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>434421760} gitlab # [ 360.834376] sidekiq-cluster[3455]: W, [2026-10-02T16:23:23.684451 #3455] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 360.900371] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:23:23.750Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 364.599898] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:23:27.449Z","message":"GitLab reliable fetch activated!"} gitlab # [ 364.600403] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:23:27.449Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 403.298972] postgres[4371]: [4371] ERROR: relation "table_without_models" does not exist gitlab # [ 403.301422] postgres[4371]: [4371] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 408.446254] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:11.289Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 408.446564] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:11.289Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} gitlab # [ 408.446643] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:11.289Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 408.446702] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:11.289Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 408.446762] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:11.291Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 82.98 seconds) gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master gitlab # [ 408.929421] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:24:11 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 81 gitlab # [ 409.054793] gitaly[948]: time="2026-10-02T16:24:11.904Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPWBR4KXP5RS5K63NE4NM1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:24:11.892" grpc.time_ms=11.999 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 409.077280] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:24:11 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.55.0" 124 gitlab # Enumerating objects: 5, done. gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. gitlab # Delta compression using up to 4 threads gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 136.00 KiB/s, done. gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) gitlab # [ 409.427049] gitaly[948]: time="2026-10-02T16:24:12.277Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPWBXHP5E3W2MVD5YBH1GC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-10-02T16:24:42.266" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-10-02T16:24:12.266" grpc.time_ms=10.582 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 409.458808] gitaly[948]: time="2026-10-02T16:24:12.309Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M3YPWBXHP5E3W2MVD5YBH1GC duration_ms=177ns method=POST pid=948 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 409.516530] gitaly[948]: time="2026-10-02T16:24:12.366Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M3YPWBXHP5E3W2MVD5YBH1GC duration_ms=57ns method=POST pid=948 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 409.524001] gitaly[948]: time="2026-10-02T16:24:12.367Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPWBXHP5E3W2MVD5YBH1GC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:24:12.129" grpc.time_ms=237.818 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 409.560783] gitaly[948]: time="2026-10-02T16:24:12.411Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPWBXHP5E3W2MVD5YBH1GC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:24:12.410" grpc.time_ms=0.879 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 409.617409] gitaly[948]: time="2026-10-02T16:24:12.467Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPWBXHP5E3W2MVD5YBH1GC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:24:12.466" grpc.time_ms=0.683 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 409.670118] gitaly[948]: time="2026-10-02T16:24:12.520Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPWBXHP5E3W2MVD5YBH1GC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:24:12.519" grpc.time_ms=0.688 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 409.913580] gitaly[948]: time="2026-10-02T16:24:12.762Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M3YPWBXHP5E3W2MVD5YBH1GC duration_ms=184ns method=POST pid=948 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 409.922195] gitaly[948]: time="2026-10-02T16:24:12.763Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPWBXHP5E3W2MVD5YBH1GC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:24:12.576" grpc.time_ms=186.528 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # To http://gitlab/root/test-flake.git gitlab # [ 409.939616] gitaly[948]: time="2026-10-02T16:24:12.777Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M3YPWBXHP5E3W2MVD5YBH1GC diskcache=6516b743-5db5-4b5e-968d-88522b12808b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:24:12.058" pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # e804315..fcd155f master -> master gitlab # [ 409.956848] gitlab-workhorse[885]: time="2026-10-02T16:24:12Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 409.959948] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:24:12 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 825 gitlab # [ 409.963360] gitaly[948]: time="2026-10-02T16:24:12.777Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=252 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=851968 command.minflt=13254 command.oublock=32 command.real_time_ms=713 command.spawn_token_fork_ms=1 command.system_time_ms=160 command.user_time_ms=92 component=gitaly.StreamServerInterceptor correlation_id=01M3YPWBXHP5E3W2MVD5YBH1GC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:24:12.058" grpc.time_ms=718.509 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.20 seconds) gitlab: must succeed: git -C /tmp/test-flake rev-parse master gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a' gitlab # [ 410.131243] nixbot[3579]: INFO: ::ffff:127.0.0.1:54932 - "GET /api/repos/gitlab/root/test-flake/builds?commit=fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a', in 0.13 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a' gitlab # [ 411.166921] nixbot[3579]: INFO: ::ffff:127.0.0.1:54238 - "GET /api/repos/gitlab/root/test-flake/builds?commit=fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a', in 0.03 seconds) gitlab # [ 411.254522] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:14.103Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"53affdc2fc6613094ba86f47","created_at":"2026-10-02T16:22:37.251Z","trace_propagation_headers":{"sentry-trace":"be38c2c065aa4c45bac1ec86b4a0eb4c-17bfd42ae6e54acb","baggage":"sentry-trace_id=be38c2c065aa4c45bac1ec86b4a0eb4c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPSEG28GDCBM1JFKV7V93S","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:37.294Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-53affdc2fc6613094ba86f47: start","job_status":"start","queue_duration_s":96.809496,"scheduling_latency_s":96.809563} gitlab # [ 411.513699] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:14.363Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"53affdc2fc6613094ba86f47","created_at":"2026-10-02T16:22:37.251Z","trace_propagation_headers":{"sentry-trace":"be38c2c065aa4c45bac1ec86b4a0eb4c-17bfd42ae6e54acb","baggage":"sentry-trace_id=be38c2c065aa4c45bac1ec86b4a0eb4c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPSEG28GDCBM1JFKV7V93S","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:37.294Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-53affdc2fc6613094ba86f47: done: 0.259859 sec","job_status":"done","queue_duration_s":96.809496,"scheduling_latency_s":96.809563,"redis_calls":23,"redis_duration_s":0.006668,"redis_read_bytes":910,"redis_write_bytes":2540,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.00341,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001941,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001317,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.140008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.13864924,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.259859,"completed_at":"2026-10-02T16:24:14.363Z","load_balancing_strategy":"primary","db_duration_s":0.069483,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 411.581294] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:14.365Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"dbc172e13e8626fe552e4aa5","created_at":"2026-10-02T16:22:37.308Z","trace_propagation_headers":{"sentry-trace":"be38c2c065aa4c45bac1ec86b4a0eb4c-17bfd42ae6e54acb","baggage":"sentry-trace_id=be38c2c065aa4c45bac1ec86b4a0eb4c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPSEG28GDCBM1JFKV7V93S","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:37.314Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-dbc172e13e8626fe552e4aa5: start","job_status":"start","queue_duration_s":97.050465,"scheduling_latency_s":97.050499} gitlab # [ 411.653459] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:14.503Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"dbc172e13e8626fe552e4aa5","created_at":"2026-10-02T16:22:37.308Z","trace_propagation_headers":{"sentry-trace":"be38c2c065aa4c45bac1ec86b4a0eb4c-17bfd42ae6e54acb","baggage":"sentry-trace_id=be38c2c065aa4c45bac1ec86b4a0eb4c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPSEG28GDCBM1JFKV7V93S","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:37.314Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-dbc172e13e8626fe552e4aa5: done: 0.137663 sec","job_status":"done","queue_duration_s":97.050465,"scheduling_latency_s":97.050499,"redis_calls":14,"redis_duration_s":0.003559,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000769,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001644,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001146,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.106141,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.10427742,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.137663,"completed_at":"2026-10-02T16:24:14.502Z","load_balancing_strategy":"primary","db_duration_s":0.023437,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 411.719858] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:14.505Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"edad5449df2b898262baa58e","created_at":"2026-10-02T16:22:37.854Z","trace_propagation_headers":{"sentry-trace":"be38c2c065aa4c45bac1ec86b4a0eb4c-17bfd42ae6e54acb","baggage":"sentry-trace_id=be38c2c065aa4c45bac1ec86b4a0eb4c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPSEG28GDCBM1JFKV7V93S","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:37.861Z","job_size_bytes":5,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-edad5449df2b898262baa58e: start","job_status":"start","queue_duration_s":96.643733,"scheduling_latency_s":96.643757} gitlab # [ 411.861280] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:14.710Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"edad5449df2b898262baa58e","created_at":"2026-10-02T16:22:37.854Z","trace_propagation_headers":{"sentry-trace":"be38c2c065aa4c45bac1ec86b4a0eb4c-17bfd42ae6e54acb","baggage":"sentry-trace_id=be38c2c065aa4c45bac1ec86b4a0eb4c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPSEG28GDCBM1JFKV7V93S","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:37.861Z","job_size_bytes":5,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-edad5449df2b898262baa58e: done: 0.205724 sec","job_status":"done","queue_duration_s":96.643733,"scheduling_latency_s":96.643757,"redis_calls":12,"redis_duration_s":0.002823,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000712,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001682,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000429,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.169133,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032934,"duration_s":0.205724,"completed_at":"2026-10-02T16:24:14.710Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.029984,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 411.931084] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:14.712Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"a5d86b000336c6bb8db2dbe0","created_at":"2026-10-02T16:22:37.892Z","trace_propagation_headers":{"sentry-trace":"be38c2c065aa4c45bac1ec86b4a0eb4c-17bfd42ae6e54acb","baggage":"sentry-trace_id=be38c2c065aa4c45bac1ec86b4a0eb4c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPSEG28GDCBM1JFKV7V93S","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:37.897Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-a5d86b000336c6bb8db2dbe0: start","job_status":"start","queue_duration_s":96.815272,"scheduling_latency_s":96.815299} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a' gitlab # [ 412.138166] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:14.987Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"a5d86b000336c6bb8db2dbe0","created_at":"2026-10-02T16:22:37.892Z","trace_propagation_headers":{"sentry-trace":"be38c2c065aa4c45bac1ec86b4a0eb4c-17bfd42ae6e54acb","baggage":"sentry-trace_id=be38c2c065aa4c45bac1ec86b4a0eb4c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPSEG28GDCBM1JFKV7V93S","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:37.897Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-a5d86b000336c6bb8db2dbe0: done: 0.27459 sec","job_status":"done","queue_duration_s":96.815272,"scheduling_latency_s":96.815299,"redis_calls":11,"redis_duration_s":0.002891,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000649,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001716,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000526,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.224,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.224,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.202609,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0006421,"duration_s":0.27459,"completed_at":"2026-10-02T16:24:14.987Z","load_balancing_strategy":"primary","db_duration_s":0.07635,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 412.208365] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:14.990Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"15e4481e3e29a2312a78741a","created_at":"2026-10-02T16:22:37.901Z","trace_propagation_headers":{"sentry-trace":"be38c2c065aa4c45bac1ec86b4a0eb4c-17bfd42ae6e54acb","baggage":"sentry-trace_id=be38c2c065aa4c45bac1ec86b4a0eb4c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPSEG28GDCBM1JFKV7V93S","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:37.906Z","job_size_bytes":89,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-15e4481e3e29a2312a78741a: start","job_status":"start","queue_duration_s":97.083791,"scheduling_latency_s":97.083826} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a', in 0.07 seconds) gitlab # [ 412.233459] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:15.047Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"15e4481e3e29a2312a78741a","created_at":"2026-10-02T16:22:37.901Z","trace_propagation_headers":{"sentry-trace":"be38c2c065aa4c45bac1ec86b4a0eb4c-17bfd42ae6e54acb","baggage":"sentry-trace_id=be38c2c065aa4c45bac1ec86b4a0eb4c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPSEG28GDCBM1JFKV7V93S","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:37.906Z","job_size_bytes":89,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-15e4481e3e29a2312a78741a: done: 0.057569 sec","job_status":"done","queue_duration_s":97.083791,"scheduling_latency_s":97.083826,"redis_calls":11,"redis_duration_s":0.002953,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00087,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001537,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000546,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00040946,"duration_s":0.057569,"completed_at":"2026-10-02T16:24:15.047Z","load_balancing_strategy":"primary","db_duration_s":0.006349,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 412.290814] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:15.049Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 e8043152c9073e6f344ce0861c23c8506ac05bcf refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"3fb27fb04b2cb502181d7090","created_at":"2026-10-02T16:22:45.678Z","trace_propagation_headers":{"sentry-trace":"d2e7adda1758473ca22e587467c0c96e-6c3b8d818bab4a00","baggage":"sentry-trace_id=d2e7adda1758473ca22e587467c0c96e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPSQ0EAB5XT7C1JC8BM5F5","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:45.686Z","job_size_bytes":185,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-3fb27fb04b2cb502181d7090: start","job_status":"start","queue_duration_s":89.363597,"scheduling_latency_s":89.36363} gitlab # [ 412.318377] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:15.077Z","message":"changes: \"0000000000000000000000000000000000000000 e8043152c9073e6f344ce0861c23c8506ac05bcf refs/heads/master\\n\""} gitlab # [ 412.322945] gitaly[948]: time="2026-10-02T16:24:15.131Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:24:25.000" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:24:15.121" grpc.time_ms=10.591 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 412.341345] nixbot[3579]: INFO: ::ffff:127.0.0.1:54252 - "GET /api/repos/gitlab/root/test-flake/builds?commit=fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a HTTP/1.1" 200 OK gitlab # [ 412.343803] gitaly[948]: time="2026-10-02T16:24:15.152Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T16:24:25.019" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:24:15.139" grpc.time_ms=12.001 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 412.390396] gitaly[948]: time="2026-10-02T16:24:15.240Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-10-02T16:24:25.107" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:24:15.227" grpc.time_ms=12.538 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 412.442776] gitaly[948]: time="2026-10-02T16:24:15.292Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3937 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:24:15.292" grpc.time_ms=0.569 pid=948 span.kind=server system=grpc gitlab # [ 412.474959] gitaly[948]: time="2026-10-02T16:24:15.325Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3939 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:24:15.324" grpc.time_ms=0.71 pid=948 span.kind=server system=grpc gitlab # [ 412.491342] gitaly[948]: time="2026-10-02T16:24:15.334Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 diskcache=0d920089-7acb-4060-8be2-4b2fe4945bac grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T16:24:25.138" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:24:15.258" pid=948 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 412.506435] gitaly[948]: time="2026-10-02T16:24:15.335Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=76 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5058 command.oublock=8 command.real_time_ms=70 command.spawn_token_fork_ms=0 command.system_time_ms=37 command.user_time_ms=39 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T16:24:25.138" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:24:15.258" grpc.time_ms=76.333 pid=948 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 412.523483] gitaly[948]: time="2026-10-02T16:24:15.356Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:24:25.235" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:24:15.355" grpc.time_ms=0.241 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 412.535704] gitaly[948]: time="2026-10-02T16:24:15.379Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=858 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-10-02T16:24:25.242" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:24:15.363" grpc.time_ms=15.932 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 412.659316] gitaly[948]: time="2026-10-02T16:24:15.509Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=312 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-10-02T16:24:45.000" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:15.501" grpc.time_ms=7.907 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 412.700890] gitaly[948]: time="2026-10-02T16:24:15.551Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-02T16:24:25.009" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:24:15.550" grpc.time_ms=0.421 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 412.718043] gitaly[948]: time="2026-10-02T16:24:15.562Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=315 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-10-02T16:24:25.014" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-10-02T16:24:15.554" grpc.time_ms=7.821 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 412.744337] gitaly[948]: time="2026-10-02T16:24:15.594Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:24:25.042" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:24:15.582" grpc.time_ms=11.6 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 412.765648] gitaly[948]: time="2026-10-02T16:24:15.613Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:24:25.062" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:24:15.602" grpc.time_ms=11.45 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 412.798067] gitaly[948]: time="2026-10-02T16:24:15.648Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=880 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-02T16:24:45.129" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:15.630" grpc.time_ms=18.08 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a' gitlab # [ 413.276909] nixbot[3579]: INFO: ::ffff:127.0.0.1:54262 - "GET /api/repos/gitlab/root/test-flake/builds?commit=fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a', in 0.04 seconds) gitlab # [ 413.328187] gitaly[948]: time="2026-10-02T16:24:16.177Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-10-02T16:24:26.055" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:16.175" grpc.time_ms=1.997 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 413.380616] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:16.229Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 e8043152c9073e6f344ce0861c23c8506ac05bcf refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"3fb27fb04b2cb502181d7090","created_at":"2026-10-02T16:22:45.678Z","trace_propagation_headers":{"sentry-trace":"d2e7adda1758473ca22e587467c0c96e-6c3b8d818bab4a00","baggage":"sentry-trace_id=d2e7adda1758473ca22e587467c0c96e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPSQ0EAB5XT7C1JC8BM5F5","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:22:45.686Z","job_size_bytes":185,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-3fb27fb04b2cb502181d7090: done: 1.179375 sec","job_status":"done","queue_duration_s":89.363597,"scheduling_latency_s":89.36363,"gitaly_calls":13,"gitaly_duration_s":0.239362,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.051678999999999996,"redis_read_bytes":2103,"redis_write_bytes":21751,"redis_cache_calls":15,"redis_cache_duration_s":0.004364,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.007161,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001919,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7839,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.007514,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.018649,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.012072,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.065,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.275,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.309,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.065,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.844472,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0009923,"exclusive_lock_hold_duration_s":0.03768492,"duration_s":1.179375,"completed_at":"2026-10-02T16:24:16.229Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.110567,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 413.453667] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:16.231Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"08444a7a03b3642d749501f7","created_at":"2026-10-02T16:22:51.539Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-3c47707a353a405f","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:51.569Z","job_size_bytes":65,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-08444a7a03b3642d749501f7: start","job_status":"start","queue_duration_s":84.662386,"scheduling_latency_s":84.662416} gitlab # [ 413.509937] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:16.359Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"08444a7a03b3642d749501f7","created_at":"2026-10-02T16:22:51.539Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-3c47707a353a405f","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:51.569Z","job_size_bytes":65,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-08444a7a03b3642d749501f7: done: 0.127862 sec","job_status":"done","queue_duration_s":84.662386,"scheduling_latency_s":84.662416,"redis_calls":16,"redis_duration_s":0.004997,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002713,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001772,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000512,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.093078,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004441,"duration_s":0.127862,"completed_at":"2026-10-02T16:24:16.359Z","load_balancing_strategy":"primary","db_duration_s":0.025907,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 413.575175] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:16.361Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"6bb1d84fc784ae26ab98c0a0","created_at":"2026-10-02T16:22:51.572Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-3c47707a353a405f","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:51.585Z","job_size_bytes":65,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-6bb1d84fc784ae26ab98c0a0: start","job_status":"start","queue_duration_s":84.776705,"scheduling_latency_s":84.776738} gitlab # [ 413.793822] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:16.643Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"6bb1d84fc784ae26ab98c0a0","created_at":"2026-10-02T16:22:51.572Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-3c47707a353a405f","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:51.585Z","job_size_bytes":65,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-6bb1d84fc784ae26ab98c0a0: done: 0.281605 sec","job_status":"done","queue_duration_s":84.776705,"scheduling_latency_s":84.776738,"redis_calls":21,"redis_duration_s":0.005172,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.002223,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001585,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001364,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.235719,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00043524,"duration_s":0.281605,"completed_at":"2026-10-02T16:24:16.643Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.034166,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 413.863954] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:16.645Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"03e6845d2cf156182a61e310","created_at":"2026-10-02T16:22:52.017Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-3c47707a353a405f","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:52.028Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-03e6845d2cf156182a61e310: start","job_status":"start","queue_duration_s":84.616755,"scheduling_latency_s":84.616783} gitlab # [ 414.025657] gitaly[948]: time="2026-10-02T16:24:16.875Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPSWP0JVJQE3Z8R2DBBT3Q grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:24:16.875" grpc.time_ms=0.604 pid=948 span.kind=server system=grpc gitlab # [ 414.059583] gitaly[948]: time="2026-10-02T16:24:16.909Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPSWP0JVJQE3Z8R2DBBT3Q grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:24:16.909" grpc.time_ms=0.618 pid=948 span.kind=server system=grpc gitlab # [ 414.092841] gitaly[948]: time="2026-10-02T16:24:16.943Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPSWP0JVJQE3Z8R2DBBT3Q grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:24:16.942" grpc.time_ms=0.639 pid=948 span.kind=server system=grpc gitlab # [ 414.110318] gitaly[948]: time="2026-10-02T16:24:16.951Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSWP0JVJQE3Z8R2DBBT3Q diskcache=f726fa6d-aa6d-4d3a-aa4f-e9418d78c2fe grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T16:24:46.339" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:24:16.839" pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 414.125032] gitaly[948]: time="2026-10-02T16:24:16.952Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=125 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7608 command.oublock=0 command.real_time_ms=107 command.spawn_token_fork_ms=0 command.system_time_ms=66 command.user_time_ms=59 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSWP0JVJQE3Z8R2DBBT3Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T16:24:46.339" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:24:16.839" grpc.time_ms=112.205 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a' gitlab # [ 414.271887] gitaly[948]: time="2026-10-02T16:24:17.122Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPSWP0JVJQE3Z8R2DBBT3Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-02T16:24:47.020" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:17.120" grpc.time_ms=1.175 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 414.316079] nixbot[3579]: INFO: ::ffff:127.0.0.1:54274 - "GET /api/repos/gitlab/root/test-flake/builds?commit=fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a', in 0.04 seconds) gitlab # [ 414.333205] gitaly[948]: time="2026-10-02T16:24:17.182Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSWP0JVJQE3Z8R2DBBT3Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:24:47.081" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:17.181" grpc.time_ms=1.041 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 414.362420] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:17.211Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"03e6845d2cf156182a61e310","created_at":"2026-10-02T16:22:52.017Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-3c47707a353a405f","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T16:22:52.028Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-03e6845d2cf156182a61e310: done: 0.566313 sec","job_status":"done","queue_duration_s":84.616755,"scheduling_latency_s":84.616783,"gitaly_calls":3,"gitaly_duration_s":0.123812,"redis_calls":117,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.030613,"redis_read_bytes":455,"redis_write_bytes":28003,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001551,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001596,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.010897,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":25,"redis_queues_duration_s":0.004212,"redis_queues_read_bytes":34,"redis_queues_write_bytes":14497,"redis_queues_metadata_calls":21,"redis_queues_metadata_duration_s":0.007233,"redis_queues_metadata_read_bytes":32,"redis_queues_metadata_write_bytes":3370,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000761,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000531,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002281,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.068,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.029,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.034,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.035,"db_ci_duration_s":0.034,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.35408,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.48216322,"duration_s":0.566313,"completed_at":"2026-10-02T16:24:17.211Z","load_balancing_strategy":"primary","db_duration_s":0.081286,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 414.436395] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:17.214Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","e8043152c9073e6f344ce0861c23c8506ac05bcf fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"6d00909b56ad5f369a7a309c","created_at":"2026-10-02T16:24:12.681Z","trace_propagation_headers":{"sentry-trace":"29fc2f409bb0475398c18933b7898801-7f33719908eb40ef","baggage":"sentry-trace_id=29fc2f409bb0475398c18933b7898801,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPWBXHP5E3W2MVD5YBH1GC","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:12.688Z","job_size_bytes":185,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-6d00909b56ad5f369a7a309c: start","job_status":"start","queue_duration_s":4.525217,"scheduling_latency_s":4.525249} gitlab # [ 414.453961] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:17.233Z","message":"changes: \"e8043152c9073e6f344ce0861c23c8506ac05bcf fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a refs/heads/master\\n\""} gitlab # [ 414.456501] gitaly[948]: time="2026-10-02T16:24:17.270Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPWBXHP5E3W2MVD5YBH1GC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:24:27.139" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:24:17.259" grpc.time_ms=11.104 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 414.470130] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:17.295Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"8c9311912ee5e6212cfbba0a","created_at":"2026-10-02T16:24:17.292Z","trace_propagation_headers":{"sentry-trace":"29fc2f409bb0475398c18933b7898801-24ebb72b6f724c76","baggage":"sentry-trace_id=29fc2f409bb0475398c18933b7898801,sentry-release=Unknown"},"correlation_id":"01M3YPWBXHP5E3W2MVD5YBH1GC","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"ffb6006a1e1778f604f26393","job_size_bytes":46,"pid":3455,"job_status":"deduplicated","message":"ProjectCacheWorker JID-8c9311912ee5e6212cfbba0a: deduplicated: until executing","deduplication.type":"until executing"} gitlab # [ 414.485597] gitaly[948]: time="2026-10-02T16:24:17.287Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWBXHP5E3W2MVD5YBH1GC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T16:24:27.158" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:24:17.278" grpc.time_ms=9.298 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 414.499378] gitaly[948]: time="2026-10-02T16:24:17.325Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YPWBXHP5E3W2MVD5YBH1GC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-10-02T16:24:27.195" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:24:17.315" grpc.time_ms=10.512 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 414.514435] gitaly[948]: time="2026-10-02T16:24:17.344Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=597 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3YPWBXHP5E3W2MVD5YBH1GC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-10-02T16:24:47.232" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:17.333" grpc.time_ms=11.051 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 414.530261] gitaly[948]: time="2026-10-02T16:24:17.368Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=318 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M3YPWBXHP5E3W2MVD5YBH1GC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-10-02T16:24:27.241" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-10-02T16:24:17.361" grpc.time_ms=6.654 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 414.543954] gitaly[948]: time="2026-10-02T16:24:17.394Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPWBXHP5E3W2MVD5YBH1GC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:24:27.262" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:24:17.383" grpc.time_ms=10.931 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 414.659957] gitaly[948]: time="2026-10-02T16:24:17.510Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPWBXHP5E3W2MVD5YBH1GC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-10-02T16:24:27.008" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:17.508" grpc.time_ms=1.584 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 414.687883] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:17.537Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","e8043152c9073e6f344ce0861c23c8506ac05bcf fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"6d00909b56ad5f369a7a309c","created_at":"2026-10-02T16:24:12.681Z","trace_propagation_headers":{"sentry-trace":"29fc2f409bb0475398c18933b7898801-7f33719908eb40ef","baggage":"sentry-trace_id=29fc2f409bb0475398c18933b7898801,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPWBXHP5E3W2MVD5YBH1GC","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:12.688Z","job_size_bytes":185,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-6d00909b56ad5f369a7a309c: done: 0.323322 sec","job_status":"done","queue_duration_s":4.525217,"scheduling_latency_s":4.525249,"gitaly_calls":7,"gitaly_duration_s":0.085276,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.022657,"redis_read_bytes":501,"redis_write_bytes":11943,"redis_cache_calls":9,"redis_cache_duration_s":0.002727,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.001057,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5720,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.004168,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.007959,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.006746,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.025,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.025,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.212819,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00116766,"exclusive_lock_hold_duration_s":0.02520182,"duration_s":0.323322,"completed_at":"2026-10-02T16:24:17.537Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.031548,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 414.762838] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:17.539Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"ffb6006a1e1778f604f26393","created_at":"2026-10-02T16:24:15.156Z","trace_propagation_headers":{"sentry-trace":"d2e7adda1758473ca22e587467c0c96e-b078bbbaebb34680","baggage":"sentry-trace_id=d2e7adda1758473ca22e587467c0c96e,sentry-release=Unknown"},"correlation_id":"01M3YPSQ0EAB5XT7C1JC8BM5F5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:15.184Z","job_size_bytes":46,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-ffb6006a1e1778f604f26393: start","job_status":"start","queue_duration_s":2.35522,"scheduling_latency_s":2.355253} gitlab # [ 414.788395] gitaly[948]: time="2026-10-02T16:24:17.611Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-10-02T22:24:17.604" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:24:17.604" grpc.time_ms=5.921 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 414.805513] gitaly[948]: time="2026-10-02T16:24:17.634Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=597 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-10-02T16:24:47.122" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:17.622" grpc.time_ms=11.097 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 414.825941] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:17.672Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"ffb6006a1e1778f604f26393","created_at":"2026-10-02T16:24:15.156Z","trace_propagation_headers":{"sentry-trace":"d2e7adda1758473ca22e587467c0c96e-b078bbbaebb34680","baggage":"sentry-trace_id=d2e7adda1758473ca22e587467c0c96e,sentry-release=Unknown"},"correlation_id":"01M3YPSQ0EAB5XT7C1JC8BM5F5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:15.184Z","job_size_bytes":46,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-ffb6006a1e1778f604f26393: done: 0.133121 sec","job_status":"done","queue_duration_s":2.35522,"scheduling_latency_s":2.355253,"gitaly_calls":2,"gitaly_duration_s":0.023297,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.008162,"redis_read_bytes":69,"redis_write_bytes":4887,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001502,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.001436,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2115,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001684,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.002733,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000807,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.088441,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003326,"duration_s":0.133121,"completed_at":"2026-10-02T16:24:17.672Z","load_balancing_strategy":"primary","db_duration_s":0.014028,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 414.882301] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:17.674Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"446f480c7fd03c4db13bc111","created_at":"2026-10-02T16:24:15.210Z","trace_propagation_headers":{"sentry-trace":"d2e7adda1758473ca22e587467c0c96e-b078bbbaebb34680","baggage":"sentry-trace_id=d2e7adda1758473ca22e587467c0c96e,sentry-release=Unknown"},"correlation_id":"01M3YPSQ0EAB5XT7C1JC8BM5F5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:15.221Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-446f480c7fd03c4db13bc111: start","job_status":"start","queue_duration_s":2.45342,"scheduling_latency_s":2.453447} gitlab # [ 414.901565] gitaly[948]: time="2026-10-02T16:24:17.728Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 diskcache=5ed91a37-82be-4946-92a0-ff056cebd5c4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-02T22:24:17.701" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:17.701" pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 414.912383] gitaly[948]: time="2026-10-02T16:24:17.729Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1171 command.oublock=0 command.real_time_ms=33 command.spawn_token_fork_ms=1 command.system_time_ms=12 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-02T22:24:17.701" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:17.701" grpc.time_ms=26.896 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 414.928349] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:17.775Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"446f480c7fd03c4db13bc111","created_at":"2026-10-02T16:24:15.210Z","trace_propagation_headers":{"sentry-trace":"d2e7adda1758473ca22e587467c0c96e-b078bbbaebb34680","baggage":"sentry-trace_id=d2e7adda1758473ca22e587467c0c96e,sentry-release=Unknown"},"correlation_id":"01M3YPSQ0EAB5XT7C1JC8BM5F5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:15.221Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-446f480c7fd03c4db13bc111: done: 0.101207 sec","job_status":"done","queue_duration_s":2.45342,"scheduling_latency_s":2.453447,"gitaly_calls":1,"gitaly_duration_s":0.029973,"redis_calls":15,"redis_duration_s":0.0033799999999999998,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00046,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001251,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000516,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001153,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.013,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.024,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.048215,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.07397480000000001,"duration_s":0.101207,"completed_at":"2026-10-02T16:24:17.775Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018574,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 414.977631] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:17.777Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"b848bb148ea841e797676419","created_at":"2026-10-02T16:24:15.348Z","trace_propagation_headers":{"sentry-trace":"d2e7adda1758473ca22e587467c0c96e-b078bbbaebb34680","baggage":"sentry-trace_id=d2e7adda1758473ca22e587467c0c96e,sentry-release=Unknown"},"correlation_id":"01M3YPSQ0EAB5XT7C1JC8BM5F5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:15.351Z","job_size_bytes":23,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-b848bb148ea841e797676419: start","job_status":"start","queue_duration_s":2.425513,"scheduling_latency_s":2.425538} gitlab # [ 414.993546] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:17.829Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"b848bb148ea841e797676419","created_at":"2026-10-02T16:24:15.348Z","trace_propagation_headers":{"sentry-trace":"d2e7adda1758473ca22e587467c0c96e-b078bbbaebb34680","baggage":"sentry-trace_id=d2e7adda1758473ca22e587467c0c96e,sentry-release=Unknown"},"correlation_id":"01M3YPSQ0EAB5XT7C1JC8BM5F5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:15.351Z","job_size_bytes":23,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-b848bb148ea841e797676419: done: 0.052308 sec","job_status":"done","queue_duration_s":2.425513,"scheduling_latency_s":2.425538,"gitaly_calls":1,"gitaly_duration_s":0.013973,"redis_calls":16,"redis_duration_s":0.003254,"redis_read_bytes":52,"redis_write_bytes":2047,"redis_queues_calls":1,"redis_queues_duration_s":0.000318,"redis_queues_read_bytes":1,"redis_queues_write_bytes":978,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001037,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001345,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000554,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030826,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.052308,"completed_at":"2026-10-02T16:24:17.829Z","load_balancing_strategy":"primary","db_duration_s":0.003167,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 415.039651] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:17.831Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"fae10ab379aa069b5c5fcff4","created_at":"2026-10-02T16:24:15.983Z","trace_propagation_headers":{"sentry-trace":"d2e7adda1758473ca22e587467c0c96e-b078bbbaebb34680","baggage":"sentry-trace_id=d2e7adda1758473ca22e587467c0c96e,sentry-release=Unknown"},"correlation_id":"01M3YPSQ0EAB5XT7C1JC8BM5F5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2db44bc582605f21bd9722d5f0c367f83e4ee0b354e1413755df695b6f47f157","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:15.989Z","job_size_bytes":467,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-fae10ab379aa069b5c5fcff4: start","job_status":"start","queue_duration_s":1.841689,"scheduling_latency_s":1.841712} gitlab # [ 415.057653] gitaly[948]: time="2026-10-02T16:24:17.819Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=596 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-10-02T16:24:47.307" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:17.808" grpc.time_ms=11.328 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 415.071162] gitaly[948]: time="2026-10-02T16:24:17.860Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:24:47.359" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:17.859" grpc.time_ms=0.786 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 415.084161] gitaly[948]: time="2026-10-02T16:24:17.873Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-02T16:24:27.002" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:24:17.863" grpc.time_ms=10.405 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 415.180900] gitaly[948]: time="2026-10-02T16:24:18.031Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-02T16:24:48.029" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:18.029" grpc.time_ms=1.111 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 415.201413] gitaly[948]: time="2026-10-02T16:24:18.039Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=65 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-10-02T16:24:48.036" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:18.036" grpc.time_ms=2.906 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a' gitlab # [ 415.315150] gitaly[948]: time="2026-10-02T16:24:18.165Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:24:28.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:24:18.154" grpc.time_ms=10.848 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 415.362364] nixbot[3579]: INFO: ::ffff:127.0.0.1:54282 - "GET /api/repos/gitlab/root/test-flake/builds?commit=fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a', in 0.05 seconds) gitlab # [ 415.951292] gitaly[948]: time="2026-10-02T16:24:18.800Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=882 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-02T16:24:48.787" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:18.788" grpc.time_ms=12.592 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 415.968966] gitaly[948]: time="2026-10-02T16:24:18.819Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPSQ0EAB5XT7C1JC8BM5F5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-10-02T16:24:48.804" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:18.804" grpc.time_ms=14.225 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 416.007885] sidekiq-cluster[3455]: {"severity":"WARN","time":"2026-10-02T16:24:18.857Z","class":"CreatePipelineWorker","correlation_id":"01M3YPSQ0EAB5XT7C1JC8BM5F5","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"e8043152c9073e6f344ce0861c23c8506ac05bcf","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"e8043152c9073e6f344ce0861c23c8506ac05bcf","ref":"refs/heads/master"},"retry":0} gitlab # [ 416.020165] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:18.867Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"fae10ab379aa069b5c5fcff4","created_at":"2026-10-02T16:24:15.983Z","trace_propagation_headers":{"sentry-trace":"d2e7adda1758473ca22e587467c0c96e-b078bbbaebb34680","baggage":"sentry-trace_id=d2e7adda1758473ca22e587467c0c96e,sentry-release=Unknown"},"correlation_id":"01M3YPSQ0EAB5XT7C1JC8BM5F5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2db44bc582605f21bd9722d5f0c367f83e4ee0b354e1413755df695b6f47f157","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:15.989Z","job_size_bytes":467,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-fae10ab379aa069b5c5fcff4: done: 1.036589 sec","job_status":"done","queue_duration_s":1.841689,"scheduling_latency_s":1.841712,"gitaly_calls":7,"gitaly_duration_s":0.072376,"redis_calls":51,"redis_duration_s":0.011163,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.001079,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.004698,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001299,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003011,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001076,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.911071,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035486,"duration_s":1.036589,"completed_at":"2026-10-02T16:24:18.867Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.059736,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 416.085939] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:18.870Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"08415e5d-0852-432a-a1f6-d74721a54700\"}"],"class":"WebHookWorker","jid":"f6f179aa7ed49d8f407f6b8a","created_at":"2026-10-02T16:24:16.135Z","trace_propagation_headers":{"sentry-trace":"d2e7adda1758473ca22e587467c0c96e-b078bbbaebb34680","baggage":"sentry-trace_id=d2e7adda1758473ca22e587467c0c96e,sentry-release=Unknown"},"correlation_id":"01M3YPSQ0EAB5XT7C1JC8BM5F5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1a4ebc478024f97487943f18db846f4ef769b1f75fb995d144baa1211969a317","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:16.143Z","job_size_bytes":1751,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-f6f179aa7ed49d8f407f6b8a: start","job_status":"start","queue_duration_s":2.727124,"scheduling_latency_s":2.727154} gitlab # [ 416.180742] nixbot[3579]: INFO: ::ffff:127.0.0.1:54284 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 416.207989] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:19.057Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"08415e5d-0852-432a-a1f6-d74721a54700\"}"],"class":"WebHookWorker","jid":"f6f179aa7ed49d8f407f6b8a","created_at":"2026-10-02T16:24:16.135Z","trace_propagation_headers":{"sentry-trace":"d2e7adda1758473ca22e587467c0c96e-b078bbbaebb34680","baggage":"sentry-trace_id=d2e7adda1758473ca22e587467c0c96e,sentry-release=Unknown"},"correlation_id":"01M3YPSQ0EAB5XT7C1JC8BM5F5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1a4ebc478024f97487943f18db846f4ef769b1f75fb995d144baa1211969a317","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:16.143Z","job_size_bytes":1751,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-f6f179aa7ed49d8f407f6b8a: done: 0.187278 sec","job_status":"done","queue_duration_s":2.727124,"scheduling_latency_s":2.727154,"redis_calls":19,"redis_duration_s":0.004342,"redis_read_bytes":32,"redis_write_bytes":5366,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001083,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000463,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3728,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001905,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000891,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.06964374000000362,"cpu_s":0.103014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00060068,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.187278,"completed_at":"2026-10-02T16:24:19.057Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003503,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 416.280582] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:19.060Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"b0ccf9667099ce1177962508","created_at":"2026-10-02T16:24:16.817Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-0f29bf7e7b154717","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:16.824Z","job_size_bytes":23,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-b0ccf9667099ce1177962508: start","job_status":"start","queue_duration_s":2.235557,"scheduling_latency_s":2.235589} gitlab # [ 416.298605] gitaly[948]: time="2026-10-02T16:24:19.134Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=0 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M3YPWJVKM6C1PDB83X3T5X47 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=194 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:24:19.126" grpc.time_ms=8.661 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 416.312867] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:24:19 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 75 gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a' gitlab # [ 416.329202] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:19.178Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"b0ccf9667099ce1177962508","created_at":"2026-10-02T16:24:16.817Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-0f29bf7e7b154717","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:16.824Z","job_size_bytes":23,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-b0ccf9667099ce1177962508: done: 0.118548 sec","job_status":"done","queue_duration_s":2.235557,"scheduling_latency_s":2.235589,"redis_calls":10,"redis_duration_s":0.002685,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00049,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001635,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00056,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.092428,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000416,"duration_s":0.118548,"completed_at":"2026-10-02T16:24:19.178Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019902,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 416.388971] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:19.180Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"c31842cc916c68768387ff28","created_at":"2026-10-02T16:24:16.959Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-0f29bf7e7b154717","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:16.968Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-c31842cc916c68768387ff28: start","job_status":"start","queue_duration_s":2.212542,"scheduling_latency_s":2.212572} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a', in 0.06 seconds) gitlab # [ 416.408229] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:19.223Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"c31842cc916c68768387ff28","created_at":"2026-10-02T16:24:16.959Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-0f29bf7e7b154717","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:16.968Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-c31842cc916c68768387ff28: done: 0.042513 sec","job_status":"done","queue_duration_s":2.212542,"scheduling_latency_s":2.212572,"redis_calls":10,"redis_duration_s":0.002503,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000386,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001345,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000772,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028554,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036074,"duration_s":0.042513,"completed_at":"2026-10-02T16:24:19.223Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005894,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 416.455076] gitlab-workhorse[885]: time="2026-10-02T16:24:19Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 416.457038] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:24:19 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 73 gitlab # [ 416.458995] gitaly[948]: time="2026-10-02T16:24:19.211Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWJY3YF3S57MC1MNW2QBN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:24:19.200" pid=948 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 416.469667] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:19.225Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"29781a69186c21c0f6fd3a3f","created_at":"2026-10-02T16:24:16.988Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-0f29bf7e7b154717","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:16.996Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-29781a69186c21c0f6fd3a3f: start","job_status":"start","queue_duration_s":2.228441,"scheduling_latency_s":2.228459} gitlab # [ 416.486772] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:19.273Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"29781a69186c21c0f6fd3a3f","created_at":"2026-10-02T16:24:16.988Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-0f29bf7e7b154717","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:16.996Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-29781a69186c21c0f6fd3a3f: done: 0.04817 sec","job_status":"done","queue_duration_s":2.228441,"scheduling_latency_s":2.228459,"redis_calls":10,"redis_duration_s":0.002672,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000426,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001739,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000507,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030754,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029998,"duration_s":0.04817,"completed_at":"2026-10-02T16:24:19.273Z","load_balancing_strategy":"primary","db_duration_s":0.010355,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 416.532410] nixbot[3579]: INFO: ::ffff:127.0.0.1:54286 - "GET /api/repos/gitlab/root/test-flake/builds?commit=fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a HTTP/1.1" 200 OK gitlab # [ 416.534404] nixbot[3579]: {"ts": "2026-10-02T16:24:19.360Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"} gitlab # [ 416.537029] gitaly[948]: time="2026-10-02T16:24:19.211Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWJY3YF3S57MC1MNW2QBN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:24:19.200" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 416.550982] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:19.277Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"63ddf9a94f811417885e9be0","created_at":"2026-10-02T16:24:17.016Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-0f29bf7e7b154717","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:17.026Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-63ddf9a94f811417885e9be0: start","job_status":"start","queue_duration_s":2.250827,"scheduling_latency_s":2.25085} gitlab # [ 416.567047] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:19.329Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"63ddf9a94f811417885e9be0","created_at":"2026-10-02T16:24:17.016Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-0f29bf7e7b154717","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:17.026Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-63ddf9a94f811417885e9be0: done: 0.052288 sec","job_status":"done","queue_duration_s":2.250827,"scheduling_latency_s":2.25085,"redis_calls":13,"redis_duration_s":0.006855,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001151,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002684,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.002132,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000888,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031528,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002796,"duration_s":0.052288,"completed_at":"2026-10-02T16:24:19.329Z","load_balancing_strategy":"primary","db_duration_s":0.008719,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 416.614828] gitlab-workhorse[885]: time="2026-10-02T16:24:19Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=336 gitlab # [ 416.616821] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:24:19 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 336 "" "git/2.55.0" 105 gitlab # [ 416.618805] gitaly[948]: time="2026-10-02T16:24:19.212Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWJY3YF3S57MC1MNW2QBN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:24:19.200" grpc.time_ms=11.88 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 416.633939] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:19.330Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"c59d5edd53ba497de0966c15","created_at":"2026-10-02T16:24:17.027Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-0f29bf7e7b154717","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:17.034Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-c59d5edd53ba497de0966c15: start","job_status":"start","queue_duration_s":2.295771,"scheduling_latency_s":2.29579} gitlab # [ 416.651230] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:19.379Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"c59d5edd53ba497de0966c15","created_at":"2026-10-02T16:24:17.027Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-0f29bf7e7b154717","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:17.034Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-c59d5edd53ba497de0966c15: done: 0.048355 sec","job_status":"done","queue_duration_s":2.295771,"scheduling_latency_s":2.29579,"redis_calls":10,"redis_duration_s":0.001994,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000454,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001133,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000407,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033823,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033458,"duration_s":0.048355,"completed_at":"2026-10-02T16:24:19.379Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007226,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 416.697417] gitaly[948]: time="2026-10-02T16:24:19.319Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWK0ESP26YY70TBXH1AHF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:24:19.301" pid=948 remote_ip="2001:db8:1::1" request_sha=5697df9c7bcdb3d691beb27939b6cac0fbe97410 response_bytes=336 span.kind=server system=grpc gitlab # [ 416.708007] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:19.380Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"dcb5c9a8e8c81da2d852bae8","created_at":"2026-10-02T16:24:17.035Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-0f29bf7e7b154717","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:17.044Z","job_size_bytes":19,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-dcb5c9a8e8c81da2d852bae8: start","job_status":"start","queue_duration_s":2.336738,"scheduling_latency_s":2.33676} gitlab # [ 416.724997] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:19.422Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"dcb5c9a8e8c81da2d852bae8","created_at":"2026-10-02T16:24:17.035Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-0f29bf7e7b154717","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:17.044Z","job_size_bytes":19,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-dcb5c9a8e8c81da2d852bae8: done: 0.041092 sec","job_status":"done","queue_duration_s":2.336738,"scheduling_latency_s":2.33676,"redis_calls":11,"redis_duration_s":0.00212,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000449,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001273,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000398,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027357,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025546,"duration_s":0.041092,"completed_at":"2026-10-02T16:24:19.421Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005783,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 416.770350] gitaly[948]: time="2026-10-02T16:24:19.319Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1129 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWK0ESP26YY70TBXH1AHF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:24:19.301" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 416.785617] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:24:19 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e8043152c9073e6f344ce0861c23c8506ac05bcf HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 224 gitlab # [ 416.788269] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:19.423Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"7be1a974c531cae3b1266035","created_at":"2026-10-02T16:24:17.049Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-0f29bf7e7b154717","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:17.056Z","job_size_bytes":66,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-7be1a974c531cae3b1266035: start","job_status":"start","queue_duration_s":2.366695,"scheduling_latency_s":2.366714} gitlab # [ 416.806147] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:19.467Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"7be1a974c531cae3b1266035","created_at":"2026-10-02T16:24:17.049Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-0f29bf7e7b154717","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:17.056Z","job_size_bytes":66,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-7be1a974c531cae3b1266035: done: 0.04402 sec","job_status":"done","queue_duration_s":2.366695,"scheduling_latency_s":2.366714,"redis_calls":10,"redis_duration_s":0.005596,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000422,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004733,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000441,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025342,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034808,"duration_s":0.04402,"completed_at":"2026-10-02T16:24:19.467Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007603,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 416.852906] nixbot[3579]: {"ts": "2026-10-02T16:24:19.600Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e8043152c9073e6f344ce0861c23c8506ac05bcf \"HTTP/1.1 201 Created\""} gitlab # [ 416.855714] gitaly[948]: time="2026-10-02T16:24:19.321Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1129 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWK0ESP26YY70TBXH1AHF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:24:19.301" grpc.time_ms=18.109 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 416.871325] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:19.470Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"f49bb1af53680642848a5d9f","created_at":"2026-10-02T16:24:17.058Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-0f29bf7e7b154717","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:17.063Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-f49bb1af53680642848a5d9f: start","job_status":"start","queue_duration_s":2.40664,"scheduling_latency_s":2.406662} gitlab # [ 416.888242] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:19.519Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"f49bb1af53680642848a5d9f","created_at":"2026-10-02T16:24:17.058Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-0f29bf7e7b154717","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:17.063Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-f49bb1af53680642848a5d9f: done: 0.048959 sec","job_status":"done","queue_duration_s":2.40664,"scheduling_latency_s":2.406662,"redis_calls":10,"redis_duration_s":0.002202,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000491,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001294,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000417,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034774,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032712,"duration_s":0.048959,"completed_at":"2026-10-02T16:24:19.519Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007429,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 416.933833] gitaly[948]: time="2026-10-02T16:24:19.454Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWK5EPCP89WMZ55GKQBJ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:24:49.452" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:19.453" grpc.time_ms=0.851 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 416.947016] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:19.521Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"de0ee01b0b6d03dcb462ed24","created_at":"2026-10-02T16:24:17.071Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-0f29bf7e7b154717","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:17.077Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-de0ee01b0b6d03dcb462ed24: start","job_status":"start","queue_duration_s":2.443656,"scheduling_latency_s":2.443673} gitlab # [ 416.963977] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:19.570Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"de0ee01b0b6d03dcb462ed24","created_at":"2026-10-02T16:24:17.071Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-0f29bf7e7b154717","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:17.077Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-de0ee01b0b6d03dcb462ed24: done: 0.049271 sec","job_status":"done","queue_duration_s":2.443656,"scheduling_latency_s":2.443673,"redis_calls":10,"redis_duration_s":0.004925,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000383,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001239,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003303,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029657,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028258,"duration_s":0.049271,"completed_at":"2026-10-02T16:24:19.570Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01057,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 417.010611] nixbot[3579]: {"ts": "2026-10-02T16:24:19.790Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e8043152c9073e6f344ce0861c23c8506ac05bcf \"HTTP/1.1 201 Created\""} gitlab # [ 417.013906] gitaly[948]: time="2026-10-02T16:24:19.535Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPWK5EPCP89WMZ55GKQBJ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:24:29.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:24:19.525" grpc.time_ms=9.8 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 417.027660] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:24:19 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e8043152c9073e6f344ce0861c23c8506ac05bcf HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 182 gitlab # [ 417.031021] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:19.572Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"cfaf687ae12dde74cd85747e","created_at":"2026-10-02T16:24:17.077Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-0f29bf7e7b154717","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:17.082Z","job_size_bytes":6,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-cfaf687ae12dde74cd85747e: start","job_status":"start","queue_duration_s":2.489504,"scheduling_latency_s":2.489521} gitlab # [ 417.047495] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:19.598Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"cfaf687ae12dde74cd85747e","created_at":"2026-10-02T16:24:17.077Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-0f29bf7e7b154717","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:17.082Z","job_size_bytes":6,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-cfaf687ae12dde74cd85747e: done: 0.026008 sec","job_status":"done","queue_duration_s":2.489504,"scheduling_latency_s":2.489521,"redis_calls":10,"redis_duration_s":0.001859,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000435,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001059,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000365,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016228,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026088,"duration_s":0.026008,"completed_at":"2026-10-02T16:24:19.598Z","load_balancing_strategy":"primary","db_duration_s":0.002562,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 417.092946] gitaly[948]: time="2026-10-02T16:24:19.669Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWKCNRF2B88NK433NNA46 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:24:49.668" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:19.669" grpc.time_ms=0.549 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 417.105976] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:19.599Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"2eb454e80de93275da23925e","created_at":"2026-10-02T16:24:17.083Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-0f29bf7e7b154717","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:17.088Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-2eb454e80de93275da23925e: start","job_status":"start","queue_duration_s":2.511505,"scheduling_latency_s":2.51152} gitlab # [ 417.121735] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:19.657Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"2eb454e80de93275da23925e","created_at":"2026-10-02T16:24:17.083Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-0f29bf7e7b154717","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:17.088Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-2eb454e80de93275da23925e: done: 0.057604 sec","job_status":"done","queue_duration_s":2.511505,"scheduling_latency_s":2.51152,"redis_calls":10,"redis_duration_s":0.002281,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000571,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00125,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00046,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036896,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00059098,"duration_s":0.057604,"completed_at":"2026-10-02T16:24:19.657Z","load_balancing_strategy":"primary","db_duration_s":0.011082,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 417.165965] gitaly[948]: time="2026-10-02T16:24:19.728Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YPWKCNRF2B88NK433NNA46 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:24:29.196" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:24:19.717" grpc.time_ms=10.472 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 417.180145] nixbot[3579]: {"ts": "2026-10-02T16:24:20.015Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e8043152c9073e6f344ce0861c23c8506ac05bcf \"HTTP/1.1 201 Created\""} gitlab # [ 417.183976] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:19.662Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"6a90717f41c5528f0a1721c3","created_at":"2026-10-02T16:24:17.097Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-0f29bf7e7b154717","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:17.105Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-6a90717f41c5528f0a1721c3: start","job_status":"start","queue_duration_s":2.557687,"scheduling_latency_s":2.55771} gitlab # [ 417.200701] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:19.889Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"6a90717f41c5528f0a1721c3","created_at":"2026-10-02T16:24:17.097Z","trace_propagation_headers":{"sentry-trace":"54742e6ebbd04877a0bc83df192b7c1f-0f29bf7e7b154717","baggage":"sentry-trace_id=54742e6ebbd04877a0bc83df192b7c1f,sentry-release=Unknown"},"correlation_id":"01M3YPSWP0JVJQE3Z8R2DBBT3Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:17.105Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-6a90717f41c5528f0a1721c3: done: 0.22682 sec","job_status":"done","queue_duration_s":2.557687,"scheduling_latency_s":2.55771,"redis_calls":14,"redis_duration_s":0.006398,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000538,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00451,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00135,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.193611,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0007533,"duration_s":0.22682,"completed_at":"2026-10-02T16:24:19.889Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01967,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 417.246234] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:24:20 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e8043152c9073e6f344ce0861c23c8506ac05bcf HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 217 gitlab # [ 417.248890] gitaly[948]: time="2026-10-02T16:24:19.891Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWKJMH8QK4MYS0KPKFGGK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:24:49.890" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:19.891" grpc.time_ms=0.707 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 417.261940] gitaly[948]: time="2026-10-02T16:24:19.920Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWBXHP5E3W2MVD5YBH1GC diskcache=455849f2-a338-4131-8dd9-f8a61a200aa6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-02T22:24:19.911" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:19.912" pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 417.272251] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:19.891Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"568fe6ba1462ce145402eb1c","created_at":"2026-10-02T16:24:17.305Z","trace_propagation_headers":{"sentry-trace":"29fc2f409bb0475398c18933b7898801-24ebb72b6f724c76","baggage":"sentry-trace_id=29fc2f409bb0475398c18933b7898801,sentry-release=Unknown"},"correlation_id":"01M3YPWBXHP5E3W2MVD5YBH1GC","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"446f480c7fd03c4db13bc111","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:17.310Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-568fe6ba1462ce145402eb1c: start","job_status":"start","queue_duration_s":2.581099,"scheduling_latency_s":2.58112} gitlab # [ 417.290300] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:19.942Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"568fe6ba1462ce145402eb1c","created_at":"2026-10-02T16:24:17.305Z","trace_propagation_headers":{"sentry-trace":"29fc2f409bb0475398c18933b7898801-24ebb72b6f724c76","baggage":"sentry-trace_id=29fc2f409bb0475398c18933b7898801,sentry-release=Unknown"},"correlation_id":"01M3YPWBXHP5E3W2MVD5YBH1GC","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"446f480c7fd03c4db13bc111","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:17.310Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-568fe6ba1462ce145402eb1c: done: 0.051466 sec","job_status":"done","queue_duration_s":2.581099,"scheduling_latency_s":2.58112,"gitaly_calls":1,"gitaly_duration_s":0.01116,"redis_calls":12,"redis_duration_s":0.002422,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000963,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000497,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000962,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027382,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.03015122,"duration_s":0.051466,"completed_at":"2026-10-02T16:24:19.942Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00823,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 417.337033] gitaly[948]: time="2026-10-02T16:24:19.920Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=289 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWBXHP5E3W2MVD5YBH1GC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-02T22:24:19.911" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:19.912" grpc.time_ms=8.042 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 417.351025] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:24:20 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 70 gitlab # [ 417.353253] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:19.944Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"f5390c41604083115b6558eb","created_at":"2026-10-02T16:24:17.458Z","trace_propagation_headers":{"sentry-trace":"29fc2f409bb0475398c18933b7898801-24ebb72b6f724c76","baggage":"sentry-trace_id=29fc2f409bb0475398c18933b7898801,sentry-release=Unknown"},"correlation_id":"01M3YPWBXHP5E3W2MVD5YBH1GC","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7f49e03db8629efe5737923bd07dd19f672fc85530eeebf886c9ceedd23bb887","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:17.461Z","job_size_bytes":467,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-f5390c41604083115b6558eb: start","job_status":"start","queue_duration_s":2.482773,"scheduling_latency_s":2.482793} gitlab # [ 417.370838] gitaly[948]: time="2026-10-02T16:24:19.949Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPWKJMH8QK4MYS0KPKFGGK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:24:29.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:24:19.940" grpc.time_ms=9.144 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 417.385765] gitlab-workhorse[885]: time="2026-10-02T16:24:20Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 417.387628] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:24:20 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 75 gitlab # [ 417.389726] gitaly[948]: time="2026-10-02T16:24:19.975Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWBXHP5E3W2MVD5YBH1GC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:24:49.071" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:19.971" grpc.time_ms=3.38 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 417.402497] gitaly[948]: time="2026-10-02T16:24:19.988Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWBXHP5E3W2MVD5YBH1GC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-02T16:24:29.120" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:24:19.980" grpc.time_ms=7.935 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 417.415920] gitaly[948]: time="2026-10-02T16:24:20.037Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPWBXHP5E3W2MVD5YBH1GC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-02T16:24:50.036" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:20.036" grpc.time_ms=0.925 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a' gitlab # [ 417.429253] gitaly[948]: time="2026-10-02T16:24:20.046Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=237 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPWBXHP5E3W2MVD5YBH1GC error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-10-02T16:24:50.041" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:20.041" grpc.time_ms=4.707 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 417.445375] gitaly[948]: time="2026-10-02T16:24:20.093Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPWBXHP5E3W2MVD5YBH1GC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:24:30.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:24:20.081" grpc.time_ms=12.086 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 417.457951] gitaly[948]: time="2026-10-02T16:24:20.126Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPWKTQVEPCN1RNQ1SZP2MD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:24:20.114" grpc.time_ms=11.25 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 417.474893] gitaly[948]: time="2026-10-02T16:24:20.203Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWKX0BBBWRKZGC418WCGP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:24:20.191" pid=948 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a', in 0.06 seconds) gitlab # [ 417.485548] gitaly[948]: time="2026-10-02T16:24:20.203Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWKX0BBBWRKZGC418WCGP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:24:20.191" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 417.500069] gitaly[948]: time="2026-10-02T16:24:20.203Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWKX0BBBWRKZGC418WCGP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:24:20.191" grpc.time_ms=12.394 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 417.514960] nixbot[3579]: INFO: ::ffff:127.0.0.1:54300 - "GET /api/repos/gitlab/root/test-flake/builds?commit=fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a HTTP/1.1" 200 OK gitlab # [ 417.819441] gitaly[948]: time="2026-10-02T16:24:20.669Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=881 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3YPWBXHP5E3W2MVD5YBH1GC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-02T16:24:50.653" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:20.653" grpc.time_ms=16.007 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 417.844270] gitaly[948]: time="2026-10-02T16:24:20.686Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YPWBXHP5E3W2MVD5YBH1GC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-10-02T16:24:50.672" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:20.672" grpc.time_ms=13.596 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 417.865296] sidekiq-cluster[3455]: {"severity":"WARN","time":"2026-10-02T16:24:20.699Z","class":"CreatePipelineWorker","correlation_id":"01M3YPWBXHP5E3W2MVD5YBH1GC","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"e8043152c9073e6f344ce0861c23c8506ac05bcf","after":"fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a","ref":"refs/heads/master"},"retry":0} gitlab # [ 417.875819] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:20.705Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"f5390c41604083115b6558eb","created_at":"2026-10-02T16:24:17.458Z","trace_propagation_headers":{"sentry-trace":"29fc2f409bb0475398c18933b7898801-24ebb72b6f724c76","baggage":"sentry-trace_id=29fc2f409bb0475398c18933b7898801,sentry-release=Unknown"},"correlation_id":"01M3YPWBXHP5E3W2MVD5YBH1GC","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7f49e03db8629efe5737923bd07dd19f672fc85530eeebf886c9ceedd23bb887","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:17.461Z","job_size_bytes":467,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-f5390c41604083115b6558eb: done: 0.761241 sec","job_status":"done","queue_duration_s":2.482773,"scheduling_latency_s":2.482793,"gitaly_calls":7,"gitaly_duration_s":0.080178,"redis_calls":33,"redis_duration_s":0.005555,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000609,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001062,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002577,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001307,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.64955,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.761241,"completed_at":"2026-10-02T16:24:20.705Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.028049,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 417.934851] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:20.707Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"d675e263-19f8-43c5-b9ee-c32261a3c264\"}"],"class":"WebHookWorker","jid":"dc51a5d85dc888045acbcb83","created_at":"2026-10-02T16:24:17.485Z","trace_propagation_headers":{"sentry-trace":"29fc2f409bb0475398c18933b7898801-24ebb72b6f724c76","baggage":"sentry-trace_id=29fc2f409bb0475398c18933b7898801,sentry-release=Unknown"},"correlation_id":"01M3YPWBXHP5E3W2MVD5YBH1GC","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:56aa6c3592c56a67685b9b3ddcfce3d3b492cdaf907ba292bfe1669f9c0fb97b","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:17.488Z","job_size_bytes":1749,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-dc51a5d85dc888045acbcb83: start","job_status":"start","queue_duration_s":3.218433,"scheduling_latency_s":3.218457} gitlab # [ 417.953605] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:20.753Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"d675e263-19f8-43c5-b9ee-c32261a3c264\"}"],"class":"WebHookWorker","jid":"dc51a5d85dc888045acbcb83","created_at":"2026-10-02T16:24:17.485Z","trace_propagation_headers":{"sentry-trace":"29fc2f409bb0475398c18933b7898801-24ebb72b6f724c76","baggage":"sentry-trace_id=29fc2f409bb0475398c18933b7898801,sentry-release=Unknown"},"correlation_id":"01M3YPWBXHP5E3W2MVD5YBH1GC","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:56aa6c3592c56a67685b9b3ddcfce3d3b492cdaf907ba292bfe1669f9c0fb97b","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:17.488Z","job_size_bytes":1749,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-dc51a5d85dc888045acbcb83: done: 0.045714 sec","job_status":"done","queue_duration_s":3.218433,"scheduling_latency_s":3.218457,"redis_calls":14,"redis_duration_s":0.002881,"redis_read_bytes":25,"redis_write_bytes":4685,"redis_queues_calls":2,"redis_queues_duration_s":0.000436,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00156,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000885,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.008117659999982152,"cpu_s":0.029305,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.045714,"completed_at":"2026-10-02T16:24:20.753Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002471,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 418.004288] nixbot[3579]: INFO: ::ffff:127.0.0.1:54308 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 418.005613] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:24:20 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 72 gitlab # [ 418.007735] gitaly[948]: time="2026-10-02T16:24:20.837Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPWMGW93GZZP0RX5HYWP8X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:24:20.827" grpc.time_ms=9.275 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 418.021615] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:20.755Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"375b813cad3938379309334b","created_at":"2026-10-02T16:24:17.646Z","trace_propagation_headers":{"sentry-trace":"d2e7adda1758473ca22e587467c0c96e-f3e91c61346948f0","baggage":"sentry-trace_id=d2e7adda1758473ca22e587467c0c96e,sentry-release=Unknown"},"correlation_id":"01M3YPSQ0EAB5XT7C1JC8BM5F5","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:17.653Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-375b813cad3938379309334b: start","job_status":"start","queue_duration_s":3.101835,"scheduling_latency_s":3.101865} gitlab # [ 418.038385] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:20.815Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"375b813cad3938379309334b","created_at":"2026-10-02T16:24:17.646Z","trace_propagation_headers":{"sentry-trace":"d2e7adda1758473ca22e587467c0c96e-f3e91c61346948f0","baggage":"sentry-trace_id=d2e7adda1758473ca22e587467c0c96e,sentry-release=Unknown"},"correlation_id":"01M3YPSQ0EAB5XT7C1JC8BM5F5","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:17.653Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-375b813cad3938379309334b: done: 0.059636 sec","job_status":"done","queue_duration_s":3.101835,"scheduling_latency_s":3.101865,"redis_calls":19,"redis_duration_s":0.004119,"redis_read_bytes":21,"redis_write_bytes":4288,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000821,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000538,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2259,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001981,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000779,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043277,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00945696,"duration_s":0.059636,"completed_at":"2026-10-02T16:24:20.815Z","load_balancing_strategy":"primary","db_duration_s":0.007487,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 418.085985] gitaly[948]: time="2026-10-02T16:24:20.923Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWMK7F3DRR3JG62D134YD grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:24:20.912" pid=948 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 418.096567] gitlab-workhorse[885]: time="2026-10-02T16:24:20Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 418.098463] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:24:20 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 85 gitlab # [ 418.100466] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:20.816Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","f6f179aa7ed49d8f407f6b8a"],"class":"WebHooks::LogExecutionWorker","jid":"229480fea336751a066e65e6","created_at":"2026-10-02T16:24:19.035Z","trace_propagation_headers":{"sentry-trace":"d2e7adda1758473ca22e587467c0c96e-617fd16059614fc9","baggage":"sentry-trace_id=d2e7adda1758473ca22e587467c0c96e,sentry-release=Unknown"},"correlation_id":"01M3YPSQ0EAB5XT7C1JC8BM5F5","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5992aa464252e56ef2fb52e7ae3c2d4cb48b247b1554f175cfe691ab5c978db0","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:19.044Z","job_size_bytes":2593,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-229480fea336751a066e65e6: start","job_status":"start","queue_duration_s":1.772029,"scheduling_latency_s":1.772048} gitlab # [ 418.117696] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:20.867Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","f6f179aa7ed49d8f407f6b8a"],"class":"WebHooks::LogExecutionWorker","jid":"229480fea336751a066e65e6","created_at":"2026-10-02T16:24:19.035Z","trace_propagation_headers":{"sentry-trace":"d2e7adda1758473ca22e587467c0c96e-617fd16059614fc9","baggage":"sentry-trace_id=d2e7adda1758473ca22e587467c0c96e,sentry-release=Unknown"},"correlation_id":"01M3YPSQ0EAB5XT7C1JC8BM5F5","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5992aa464252e56ef2fb52e7ae3c2d4cb48b247b1554f175cfe691ab5c978db0","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:19.044Z","job_size_bytes":2593,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-229480fea336751a066e65e6: done: 0.050988 sec","job_status":"done","queue_duration_s":1.772029,"scheduling_latency_s":1.772048,"redis_calls":10,"redis_duration_s":0.001934,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000412,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001149,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000373,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035381,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003751,"duration_s":0.050988,"completed_at":"2026-10-02T16:24:20.867Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009427,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 418.164364] gitaly[948]: time="2026-10-02T16:24:20.924Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWMK7F3DRR3JG62D134YD grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:24:20.912" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 418.178154] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:20.869Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"9299302387845f84684fcf1c","created_at":"2026-10-02T16:24:19.572Z","trace_propagation_headers":{"sentry-trace":"5d7fe09f458248fa96c309b759612232-859b7ea673ca41a8","baggage":"sentry-trace_id=5d7fe09f458248fa96c309b759612232,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPWK5EPCP89WMZ55GKQBJ3","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:19.579Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-9299302387845f84684fcf1c: start","job_status":"start","queue_duration_s":1.290393,"scheduling_latency_s":1.290417} gitlab # [ 418.194695] gitaly[948]: time="2026-10-02T16:24:20.924Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWMK7F3DRR3JG62D134YD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:24:20.912" grpc.time_ms=11.722 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 418.211061] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:24:21 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 65 gitlab # [ 418.213410] gitaly[948]: time="2026-10-02T16:24:20.991Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPWK5EPCP89WMZ55GKQBJ3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:24:20.990" grpc.time_ms=0.651 pid=948 span.kind=server system=grpc gitlab # [ 418.223409] gitaly[948]: time="2026-10-02T16:24:21.014Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPWK5EPCP89WMZ55GKQBJ3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:24:21.013" grpc.time_ms=0.555 pid=948 span.kind=server system=grpc gitlab # [ 418.233818] gitaly[948]: time="2026-10-02T16:24:21.037Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPWMQCB5ECJKJ7HH2Y8GC2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:24:21.026" grpc.time_ms=10.88 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 418.248083] gitaly[948]: time="2026-10-02T16:24:21.045Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPWK5EPCP89WMZ55GKQBJ3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:24:21.045" grpc.time_ms=0.709 pid=948 span.kind=server system=grpc gitlab # [ 418.257976] gitaly[948]: time="2026-10-02T16:24:21.052Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWK5EPCP89WMZ55GKQBJ3 diskcache=5fd1e719-78a0-48a0-8f40-1bc0b0cd97aa grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T16:24:50.054" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:24:20.955" pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 418.268186] gitaly[948]: time="2026-10-02T16:24:21.052Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=82 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=6950 command.oublock=0 command.real_time_ms=93 command.spawn_token_fork_ms=0 command.system_time_ms=46 command.user_time_ms=36 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWK5EPCP89WMZ55GKQBJ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T16:24:50.054" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:24:20.955" grpc.time_ms=97.537 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 418.282480] gitaly[948]: time="2026-10-02T16:24:21.122Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWMSGB3GZA78XPTHGGRR2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:24:21.103" pid=948 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc gitlab # [ 418.293409] gitlab-workhorse[885]: time="2026-10-02T16:24:21Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=216 gitlab # [ 418.295350] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:24:21 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 83 gitlab # [ 418.297427] gitaly[948]: time="2026-10-02T16:24:21.123Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1101 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=1 command.system_time_ms=12 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWMSGB3GZA78XPTHGGRR2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:24:21.103" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 418.311889] gitaly[948]: time="2026-10-02T16:24:21.123Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1101 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=1 command.system_time_ms=12 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWMSGB3GZA78XPTHGGRR2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:24:21.103" grpc.time_ms=19.665 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 418.327557] gitaly[948]: time="2026-10-02T16:24:21.134Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPWK5EPCP89WMZ55GKQBJ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-02T16:24:51.132" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:21.133" grpc.time_ms=0.971 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 418.340959] gitaly[948]: time="2026-10-02T16:24:21.177Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWK5EPCP89WMZ55GKQBJ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:24:51.175" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:21.176" grpc.time_ms=0.838 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 418.353847] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.188Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"9299302387845f84684fcf1c","created_at":"2026-10-02T16:24:19.572Z","trace_propagation_headers":{"sentry-trace":"5d7fe09f458248fa96c309b759612232-859b7ea673ca41a8","baggage":"sentry-trace_id=5d7fe09f458248fa96c309b759612232,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPWK5EPCP89WMZ55GKQBJ3","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:19.579Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3455,"job_status":"rescheduled","message":"PipelineProcessWorker JID-9299302387845f84684fcf1c: rescheduled"} gitlab # [ 418.369387] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.196Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"9299302387845f84684fcf1c","created_at":"2026-10-02T16:24:19.572Z","trace_propagation_headers":{"sentry-trace":"5d7fe09f458248fa96c309b759612232-859b7ea673ca41a8","baggage":"sentry-trace_id=5d7fe09f458248fa96c309b759612232,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPWK5EPCP89WMZ55GKQBJ3","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:19.579Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-9299302387845f84684fcf1c: done: 0.326994 sec","job_status":"done","queue_duration_s":1.290393,"scheduling_latency_s":1.290417,"gitaly_calls":3,"gitaly_duration_s":0.107793,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.013111,"redis_read_bytes":148,"redis_write_bytes":18017,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000823,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000803,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.003131,"redis_queues_read_bytes":26,"redis_queues_write_bytes":12316,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.004847,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000352,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00048,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001852,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.046,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.009,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.009,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.02,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.166926,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.27864458,"duration_s":0.326994,"completed_at":"2026-10-02T16:24:21.196Z","load_balancing_strategy":"primary","db_duration_s":0.046904,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 418.423541] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.198Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","dc51a5d85dc888045acbcb83"],"class":"WebHooks::LogExecutionWorker","jid":"6b9e26769adca753b425ba1e","created_at":"2026-10-02T16:24:20.740Z","trace_propagation_headers":{"sentry-trace":"29fc2f409bb0475398c18933b7898801-207605ab037c439c","baggage":"sentry-trace_id=29fc2f409bb0475398c18933b7898801,sentry-release=Unknown"},"correlation_id":"01M3YPWBXHP5E3W2MVD5YBH1GC","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:01e5473f556474bc86faee28a908ca13fb36bf4134e66bd51983da72f04bc23f","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:20.744Z","job_size_bytes":2592,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-6b9e26769adca753b425ba1e: start","job_status":"start","queue_duration_s":0.453685,"scheduling_latency_s":0.453712} gitlab # [ 418.441638] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.227Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","dc51a5d85dc888045acbcb83"],"class":"WebHooks::LogExecutionWorker","jid":"6b9e26769adca753b425ba1e","created_at":"2026-10-02T16:24:20.740Z","trace_propagation_headers":{"sentry-trace":"29fc2f409bb0475398c18933b7898801-207605ab037c439c","baggage":"sentry-trace_id=29fc2f409bb0475398c18933b7898801,sentry-release=Unknown"},"correlation_id":"01M3YPWBXHP5E3W2MVD5YBH1GC","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:01e5473f556474bc86faee28a908ca13fb36bf4134e66bd51983da72f04bc23f","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:20.744Z","job_size_bytes":2592,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-6b9e26769adca753b425ba1e: done: 0.029443 sec","job_status":"done","queue_duration_s":0.453685,"scheduling_latency_s":0.453712,"redis_calls":7,"redis_duration_s":0.0011849999999999999,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000877,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000308,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02053,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029443,"completed_at":"2026-10-02T16:24:21.227Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003816,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a' gitlab # [ 418.486936] gitaly[948]: time="2026-10-02T16:24:21.224Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWMX36CQH5NJ0WSE420VZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:24:51.223" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:21.223" grpc.time_ms=0.813 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 418.500123] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.228Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"527378bcb4c5e94e6ab6d687","created_at":"2026-10-02T16:24:20.798Z","trace_propagation_headers":{"sentry-trace":"d2e7adda1758473ca22e587467c0c96e-73d50cc6378143d0","baggage":"sentry-trace_id=d2e7adda1758473ca22e587467c0c96e,sentry-release=Unknown"},"correlation_id":"01M3YPSQ0EAB5XT7C1JC8BM5F5","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:20.805Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-527378bcb4c5e94e6ab6d687: start","job_status":"start","queue_duration_s":0.423759,"scheduling_latency_s":0.42379} gitlab # [ 418.516897] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.338Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"527378bcb4c5e94e6ab6d687","created_at":"2026-10-02T16:24:20.798Z","trace_propagation_headers":{"sentry-trace":"d2e7adda1758473ca22e587467c0c96e-73d50cc6378143d0","baggage":"sentry-trace_id=d2e7adda1758473ca22e587467c0c96e,sentry-release=Unknown"},"correlation_id":"01M3YPSQ0EAB5XT7C1JC8BM5F5","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:20.805Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-527378bcb4c5e94e6ab6d687: done: 0.109117 sec","job_status":"done","queue_duration_s":0.423759,"scheduling_latency_s":0.42379,"redis_calls":11,"redis_duration_s":0.002426,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000389,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001626,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000411,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.080366,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00052952,"duration_s":0.109117,"completed_at":"2026-10-02T16:24:21.337Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022829,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a', in 0.07 seconds) (finished: subtest: push delivers the webhook, in 92.76 seconds) subtest: eval and build post commit statuses gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses gitlab # [ 418.563886] gitaly[948]: time="2026-10-02T16:24:21.249Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=592 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=5 command.system_time_ms=4 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M3YPWMX36CQH5NJ0WSE420VZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-10-02T16:24:51.236" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:24:21.236" grpc.time_ms=12.925 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 418.578789] nixbot[3579]: INFO: ::ffff:127.0.0.1:54324 - "GET /api/repos/gitlab/root/test-flake/builds?commit=fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a HTTP/1.1" 200 OK gitlab # [ 418.580873] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.339Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"6acfc86bfeb7d731dfa4fb9f","created_at":"2026-10-02T16:24:20.945Z","trace_propagation_headers":{"sentry-trace":"5d7fe09f458248fa96c309b759612232-d5916225b8294023","baggage":"sentry-trace_id=5d7fe09f458248fa96c309b759612232,sentry-release=Unknown"},"correlation_id":"01M3YPWK5EPCP89WMZ55GKQBJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:20.948Z","job_size_bytes":23,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-6acfc86bfeb7d731dfa4fb9f: start","job_status":"start","queue_duration_s":0.391378,"scheduling_latency_s":0.391404} gitlab # [ 418.597803] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.399Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"6acfc86bfeb7d731dfa4fb9f","created_at":"2026-10-02T16:24:20.945Z","trace_propagation_headers":{"sentry-trace":"5d7fe09f458248fa96c309b759612232-d5916225b8294023","baggage":"sentry-trace_id=5d7fe09f458248fa96c309b759612232,sentry-release=Unknown"},"correlation_id":"01M3YPWK5EPCP89WMZ55GKQBJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:20.948Z","job_size_bytes":23,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-6acfc86bfeb7d731dfa4fb9f: done: 0.059938 sec","job_status":"done","queue_duration_s":0.391378,"scheduling_latency_s":0.391404,"redis_calls":7,"redis_duration_s":0.001742,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00118,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000562,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043774,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.059938,"completed_at":"2026-10-02T16:24:21.399Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010738,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 418.642965] gitaly[948]: time="2026-10-02T16:24:21.269Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3YPWMX36CQH5NJ0WSE420VZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:24:31.119" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:24:21.259" grpc.time_ms=9.522 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 418.656830] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.401Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"47da0c0001cf5dadd319cbb1","created_at":"2026-10-02T16:24:21.064Z","trace_propagation_headers":{"sentry-trace":"5d7fe09f458248fa96c309b759612232-d5916225b8294023","baggage":"sentry-trace_id=5d7fe09f458248fa96c309b759612232,sentry-release=Unknown"},"correlation_id":"01M3YPWK5EPCP89WMZ55GKQBJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:21.068Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-47da0c0001cf5dadd319cbb1: start","job_status":"start","queue_duration_s":0.333067,"scheduling_latency_s":0.333102} gitlab # [ 418.674195] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.450Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"47da0c0001cf5dadd319cbb1","created_at":"2026-10-02T16:24:21.064Z","trace_propagation_headers":{"sentry-trace":"5d7fe09f458248fa96c309b759612232-d5916225b8294023","baggage":"sentry-trace_id=5d7fe09f458248fa96c309b759612232,sentry-release=Unknown"},"correlation_id":"01M3YPWK5EPCP89WMZ55GKQBJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:21.068Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-47da0c0001cf5dadd319cbb1: done: 0.049124 sec","job_status":"done","queue_duration_s":0.333067,"scheduling_latency_s":0.333102,"redis_calls":7,"redis_duration_s":0.002141,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001604,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000537,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033487,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.049124,"completed_at":"2026-10-02T16:24:21.450Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005223,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses, in 0.16 seconds) gitlab # [ 418.718329] gitaly[948]: time="2026-10-02T16:24:21.291Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWMX36CQH5NJ0WSE420VZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-02T16:24:31.149" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:24:21.290" grpc.time_ms=0.49 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 418.729781] nixbot[3579]: {"ts": "2026-10-02T16:24:21.512Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a \"HTTP/1.1 201 Created\""} gitlab # [ 418.732659] nixbot[3579]: {"ts": "2026-10-02T16:24:21.524Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-fcd155fee695-5"} gitlab # [ 418.735003] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.453Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"f90b8e9019986cd896dfd271","created_at":"2026-10-02T16:24:21.080Z","trace_propagation_headers":{"sentry-trace":"5d7fe09f458248fa96c309b759612232-d5916225b8294023","baggage":"sentry-trace_id=5d7fe09f458248fa96c309b759612232,sentry-release=Unknown"},"correlation_id":"01M3YPWK5EPCP89WMZ55GKQBJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:21.083Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-f90b8e9019986cd896dfd271: start","job_status":"start","queue_duration_s":0.369939,"scheduling_latency_s":0.369978} gitlab # [ 418.751500] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.504Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"f90b8e9019986cd896dfd271","created_at":"2026-10-02T16:24:21.080Z","trace_propagation_headers":{"sentry-trace":"5d7fe09f458248fa96c309b759612232-d5916225b8294023","baggage":"sentry-trace_id=5d7fe09f458248fa96c309b759612232,sentry-release=Unknown"},"correlation_id":"01M3YPWK5EPCP89WMZ55GKQBJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:21.083Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-f90b8e9019986cd896dfd271: done: 0.05052 sec","job_status":"done","queue_duration_s":0.369939,"scheduling_latency_s":0.369978,"redis_calls":7,"redis_duration_s":0.0017100000000000001,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001268,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000442,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037474,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.05052,"completed_at":"2026-10-02T16:24:21.503Z","load_balancing_strategy":"primary","db_duration_s":0.005528,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 418.795450] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:24:21 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 354 gitlab # [ 418.798274] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:24:21 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 115 gitlab # [ 418.801064] gitaly[948]: time="2026-10-02T16:24:21.307Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWMX36CQH5NJ0WSE420VZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-02T16:24:31.154" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:24:21.295" grpc.time_ms=11.929 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 418.814838] gitaly[948]: time="2026-10-02T16:24:21.352Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPWMX36CQH5NJ0WSE420VZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:24:21.350" grpc.time_ms=0.816 pid=948 span.kind=server system=grpc gitlab # [ 418.825926] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.505Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"6de0c48e888d6a674896fa15","created_at":"2026-10-02T16:24:21.094Z","trace_propagation_headers":{"sentry-trace":"5d7fe09f458248fa96c309b759612232-d5916225b8294023","baggage":"sentry-trace_id=5d7fe09f458248fa96c309b759612232,sentry-release=Unknown"},"correlation_id":"01M3YPWK5EPCP89WMZ55GKQBJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:21.097Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-6de0c48e888d6a674896fa15: start","job_status":"start","queue_duration_s":0.407966,"scheduling_latency_s":0.407998} gitlab # [ 418.842008] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.550Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"6de0c48e888d6a674896fa15","created_at":"2026-10-02T16:24:21.094Z","trace_propagation_headers":{"sentry-trace":"5d7fe09f458248fa96c309b759612232-d5916225b8294023","baggage":"sentry-trace_id=5d7fe09f458248fa96c309b759612232,sentry-release=Unknown"},"correlation_id":"01M3YPWK5EPCP89WMZ55GKQBJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:21.097Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-6de0c48e888d6a674896fa15: done: 0.045192 sec","job_status":"done","queue_duration_s":0.407966,"scheduling_latency_s":0.407998,"redis_calls":10,"redis_duration_s":0.00164,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000838,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000458,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000344,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032663,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.045192,"completed_at":"2026-10-02T16:24:21.550Z","load_balancing_strategy":"primary","db_duration_s":0.006099,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 418.886570] gitaly[948]: time="2026-10-02T16:24:21.395Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPWMX36CQH5NJ0WSE420VZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:24:21.395" grpc.time_ms=0.535 pid=948 span.kind=server system=grpc gitlab # [ 418.896598] gitaly[948]: time="2026-10-02T16:24:21.401Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWMX36CQH5NJ0WSE420VZ diskcache=7613b9fe-1deb-4bf4-aee3-8467164356af grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T16:24:31.169" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:24:21.310" pid=948 ref=refs/keep-around/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a remote_ip="2001:db8:1::1" revision=fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a span.kind=server system=grpc username=root gitlab # [ 418.908389] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.552Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"c82b5cf4d652c5b72c724bf4","created_at":"2026-10-02T16:24:21.098Z","trace_propagation_headers":{"sentry-trace":"5d7fe09f458248fa96c309b759612232-d5916225b8294023","baggage":"sentry-trace_id=5d7fe09f458248fa96c309b759612232,sentry-release=Unknown"},"correlation_id":"01M3YPWK5EPCP89WMZ55GKQBJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:21.101Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-c82b5cf4d652c5b72c724bf4: start","job_status":"start","queue_duration_s":0.450861,"scheduling_latency_s":0.450882} gitlab # [ 418.925462] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.588Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"c82b5cf4d652c5b72c724bf4","created_at":"2026-10-02T16:24:21.098Z","trace_propagation_headers":{"sentry-trace":"5d7fe09f458248fa96c309b759612232-d5916225b8294023","baggage":"sentry-trace_id=5d7fe09f458248fa96c309b759612232,sentry-release=Unknown"},"correlation_id":"01M3YPWK5EPCP89WMZ55GKQBJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:21.101Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-c82b5cf4d652c5b72c724bf4: done: 0.036066 sec","job_status":"done","queue_duration_s":0.450861,"scheduling_latency_s":0.450882,"redis_calls":7,"redis_duration_s":0.00116,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000849,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000311,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026519,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036066,"completed_at":"2026-10-02T16:24:21.588Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005688,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 418.969449] nixbot[3579]: {"ts": "2026-10-02T16:24:21.682Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-fcd155fee695-5", "argv0": "bwrap"} gitlab # [ 418.971871] gitaly[948]: time="2026-10-02T16:24:21.402Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=77 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5129 command.oublock=8 command.real_time_ms=87 command.spawn_token_fork_ms=1 command.system_time_ms=38 command.user_time_ms=39 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWMX36CQH5NJ0WSE420VZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T16:24:31.169" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:24:21.310" grpc.time_ms=91.754 pid=948 ref=refs/keep-around/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a remote_ip="2001:db8:1::1" revision=fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a span.kind=server system=grpc username=root gitlab # [ 418.987293] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.589Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"d74bb34b337f406fe7a2d8c2","created_at":"2026-10-02T16:24:21.102Z","trace_propagation_headers":{"sentry-trace":"5d7fe09f458248fa96c309b759612232-d5916225b8294023","baggage":"sentry-trace_id=5d7fe09f458248fa96c309b759612232,sentry-release=Unknown"},"correlation_id":"01M3YPWK5EPCP89WMZ55GKQBJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:21.106Z","job_size_bytes":19,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-d74bb34b337f406fe7a2d8c2: start","job_status":"start","queue_duration_s":0.483253,"scheduling_latency_s":0.483272} gitlab # [ 419.004433] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.618Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"d74bb34b337f406fe7a2d8c2","created_at":"2026-10-02T16:24:21.102Z","trace_propagation_headers":{"sentry-trace":"5d7fe09f458248fa96c309b759612232-d5916225b8294023","baggage":"sentry-trace_id=5d7fe09f458248fa96c309b759612232,sentry-release=Unknown"},"correlation_id":"01M3YPWK5EPCP89WMZ55GKQBJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:21.106Z","job_size_bytes":19,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-d74bb34b337f406fe7a2d8c2: done: 0.029107 sec","job_status":"done","queue_duration_s":0.483253,"scheduling_latency_s":0.483272,"redis_calls":8,"redis_duration_s":0.001457,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001062,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000395,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020048,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029107,"completed_at":"2026-10-02T16:24:21.618Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004169,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 419.049661] gitaly[948]: time="2026-10-02T16:24:21.406Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWMX36CQH5NJ0WSE420VZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:24:51.405" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:21.405" grpc.time_ms=0.363 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 419.063162] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.620Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"aca0f113bd018ff62a58669c","created_at":"2026-10-02T16:24:21.107Z","trace_propagation_headers":{"sentry-trace":"5d7fe09f458248fa96c309b759612232-d5916225b8294023","baggage":"sentry-trace_id=5d7fe09f458248fa96c309b759612232,sentry-release=Unknown"},"correlation_id":"01M3YPWK5EPCP89WMZ55GKQBJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:21.111Z","job_size_bytes":66,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-aca0f113bd018ff62a58669c: start","job_status":"start","queue_duration_s":0.508686,"scheduling_latency_s":0.508708} gitlab # [ 419.081134] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.646Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"aca0f113bd018ff62a58669c","created_at":"2026-10-02T16:24:21.107Z","trace_propagation_headers":{"sentry-trace":"5d7fe09f458248fa96c309b759612232-d5916225b8294023","baggage":"sentry-trace_id=5d7fe09f458248fa96c309b759612232,sentry-release=Unknown"},"correlation_id":"01M3YPWK5EPCP89WMZ55GKQBJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:21.111Z","job_size_bytes":66,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-aca0f113bd018ff62a58669c: done: 0.026893 sec","job_status":"done","queue_duration_s":0.508686,"scheduling_latency_s":0.508708,"redis_calls":7,"redis_duration_s":0.001154,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000856,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000298,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019171,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026893,"completed_at":"2026-10-02T16:24:21.646Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002913,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 419.128084] gitaly[948]: time="2026-10-02T16:24:21.525Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWN6C1EGF5Y3T9143CE22 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:24:51.023" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:21.524" grpc.time_ms=1.617 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 419.141696] nix-daemon[3748]: accepted connection from pid 4833, user nixbot gitlab # [ 419.142798] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.648Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"a42342eab20d4fd047bef80f","created_at":"2026-10-02T16:24:21.112Z","trace_propagation_headers":{"sentry-trace":"5d7fe09f458248fa96c309b759612232-d5916225b8294023","baggage":"sentry-trace_id=5d7fe09f458248fa96c309b759612232,sentry-release=Unknown"},"correlation_id":"01M3YPWK5EPCP89WMZ55GKQBJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:21.114Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-a42342eab20d4fd047bef80f: start","job_status":"start","queue_duration_s":0.533677,"scheduling_latency_s":0.533694} gitlab # [ 419.159597] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.676Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"a42342eab20d4fd047bef80f","created_at":"2026-10-02T16:24:21.112Z","trace_propagation_headers":{"sentry-trace":"5d7fe09f458248fa96c309b759612232-d5916225b8294023","baggage":"sentry-trace_id=5d7fe09f458248fa96c309b759612232,sentry-release=Unknown"},"correlation_id":"01M3YPWK5EPCP89WMZ55GKQBJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:21.114Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-a42342eab20d4fd047bef80f: done: 0.028401 sec","job_status":"done","queue_duration_s":0.533677,"scheduling_latency_s":0.533694,"redis_calls":7,"redis_duration_s":0.001054,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000778,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000276,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019241,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028401,"completed_at":"2026-10-02T16:24:21.676Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005382,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 419.204090] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.677Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"91f846a4841b8be1333abdf6","created_at":"2026-10-02T16:24:21.115Z","trace_propagation_headers":{"sentry-trace":"5d7fe09f458248fa96c309b759612232-d5916225b8294023","baggage":"sentry-trace_id=5d7fe09f458248fa96c309b759612232,sentry-release=Unknown"},"correlation_id":"01M3YPWK5EPCP89WMZ55GKQBJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:21.117Z","job_size_bytes":6,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-91f846a4841b8be1333abdf6: start","job_status":"start","queue_duration_s":0.560345,"scheduling_latency_s":0.560364} gitlab # [ 419.220412] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.692Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"91f846a4841b8be1333abdf6","created_at":"2026-10-02T16:24:21.115Z","trace_propagation_headers":{"sentry-trace":"5d7fe09f458248fa96c309b759612232-d5916225b8294023","baggage":"sentry-trace_id=5d7fe09f458248fa96c309b759612232,sentry-release=Unknown"},"correlation_id":"01M3YPWK5EPCP89WMZ55GKQBJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:21.117Z","job_size_bytes":6,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-91f846a4841b8be1333abdf6: done: 0.014769 sec","job_status":"done","queue_duration_s":0.560345,"scheduling_latency_s":0.560364,"redis_calls":7,"redis_duration_s":0.001204,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000838,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000366,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014769,"completed_at":"2026-10-02T16:24:21.692Z","load_balancing_strategy":"primary","db_duration_s":0.000497,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 419.264283] gitaly[948]: time="2026-10-02T16:24:22.004Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPWMX36CQH5NJ0WSE420VZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-02T16:24:52.002" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:22.002" grpc.time_ms=1.244 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 419.277815] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.693Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"c9d68312fc73a19f444338f1","created_at":"2026-10-02T16:24:21.118Z","trace_propagation_headers":{"sentry-trace":"5d7fe09f458248fa96c309b759612232-d5916225b8294023","baggage":"sentry-trace_id=5d7fe09f458248fa96c309b759612232,sentry-release=Unknown"},"correlation_id":"01M3YPWK5EPCP89WMZ55GKQBJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:21.120Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-c9d68312fc73a19f444338f1: start","job_status":"start","queue_duration_s":0.573613,"scheduling_latency_s":0.573629} gitlab # [ 419.293932] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.718Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"c9d68312fc73a19f444338f1","created_at":"2026-10-02T16:24:21.118Z","trace_propagation_headers":{"sentry-trace":"5d7fe09f458248fa96c309b759612232-d5916225b8294023","baggage":"sentry-trace_id=5d7fe09f458248fa96c309b759612232,sentry-release=Unknown"},"correlation_id":"01M3YPWK5EPCP89WMZ55GKQBJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:21.120Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-c9d68312fc73a19f444338f1: done: 0.024884 sec","job_status":"done","queue_duration_s":0.573613,"scheduling_latency_s":0.573629,"redis_calls":7,"redis_duration_s":0.001261,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000911,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00035,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017163,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024884,"completed_at":"2026-10-02T16:24:21.718Z","load_balancing_strategy":"primary","db_duration_s":0.003811,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 419.337702] gitaly[948]: time="2026-10-02T16:24:22.056Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWMX36CQH5NJ0WSE420VZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:24:52.054" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:22.055" grpc.time_ms=0.886 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 419.350903] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.720Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"6e5d709cbc35e0ff1dd8accc","created_at":"2026-10-02T16:24:21.188Z","trace_propagation_headers":{"sentry-trace":"5d7fe09f458248fa96c309b759612232-d5916225b8294023","baggage":"sentry-trace_id=5d7fe09f458248fa96c309b759612232,sentry-release=Unknown"},"correlation_id":"01M3YPWK5EPCP89WMZ55GKQBJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:21.190Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-6e5d709cbc35e0ff1dd8accc: start","job_status":"start","queue_duration_s":0.528982,"scheduling_latency_s":0.528998} gitlab # [ 419.367026] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.748Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"6e5d709cbc35e0ff1dd8accc","created_at":"2026-10-02T16:24:21.188Z","trace_propagation_headers":{"sentry-trace":"5d7fe09f458248fa96c309b759612232-d5916225b8294023","baggage":"sentry-trace_id=5d7fe09f458248fa96c309b759612232,sentry-release=Unknown"},"correlation_id":"01M3YPWK5EPCP89WMZ55GKQBJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:21.190Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-6e5d709cbc35e0ff1dd8accc: done: 0.028737 sec","job_status":"done","queue_duration_s":0.528982,"scheduling_latency_s":0.528998,"redis_calls":8,"redis_duration_s":0.0015249999999999999,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001181,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000344,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020197,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028737,"completed_at":"2026-10-02T16:24:21.748Z","load_balancing_strategy":"primary","db_duration_s":0.004528,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 419.410890] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.750Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"adf510534184dec5694ecbda","created_at":"2026-10-02T16:24:21.408Z","trace_propagation_headers":{"sentry-trace":"fccc11cd6c5c4d9cb6fab3cf995af8be-942e988359c7489e","baggage":"sentry-trace_id=fccc11cd6c5c4d9cb6fab3cf995af8be,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPWMX36CQH5NJ0WSE420VZ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:21.412Z","job_size_bytes":65,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-adf510534184dec5694ecbda: start","job_status":"start","queue_duration_s":0.337283,"scheduling_latency_s":0.337298} gitlab # [ 419.428607] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.782Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"adf510534184dec5694ecbda","created_at":"2026-10-02T16:24:21.408Z","trace_propagation_headers":{"sentry-trace":"fccc11cd6c5c4d9cb6fab3cf995af8be-942e988359c7489e","baggage":"sentry-trace_id=fccc11cd6c5c4d9cb6fab3cf995af8be,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPWMX36CQH5NJ0WSE420VZ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:21.412Z","job_size_bytes":65,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-adf510534184dec5694ecbda: done: 0.032172 sec","job_status":"done","queue_duration_s":0.337283,"scheduling_latency_s":0.337298,"redis_calls":7,"redis_duration_s":0.001421,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001024,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000397,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021896,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032172,"completed_at":"2026-10-02T16:24:21.782Z","load_balancing_strategy":"primary","db_duration_s":0.005783,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 419.474786] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.783Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"02a845b2d28c3ba27346fa7b","created_at":"2026-10-02T16:24:21.413Z","trace_propagation_headers":{"sentry-trace":"fccc11cd6c5c4d9cb6fab3cf995af8be-942e988359c7489e","baggage":"sentry-trace_id=fccc11cd6c5c4d9cb6fab3cf995af8be,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPWMX36CQH5NJ0WSE420VZ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:21.417Z","job_size_bytes":65,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-02a845b2d28c3ba27346fa7b: start","job_status":"start","queue_duration_s":0.36596,"scheduling_latency_s":0.365979} gitlab # [ 419.493107] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.889Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"02a845b2d28c3ba27346fa7b","created_at":"2026-10-02T16:24:21.413Z","trace_propagation_headers":{"sentry-trace":"fccc11cd6c5c4d9cb6fab3cf995af8be-942e988359c7489e","baggage":"sentry-trace_id=fccc11cd6c5c4d9cb6fab3cf995af8be,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPWMX36CQH5NJ0WSE420VZ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:21.417Z","job_size_bytes":65,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-02a845b2d28c3ba27346fa7b: done: 0.105402 sec","job_status":"done","queue_duration_s":0.36596,"scheduling_latency_s":0.365979,"redis_calls":10,"redis_duration_s":0.00304,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001762,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001278,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.071789,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.105402,"completed_at":"2026-10-02T16:24:21.889Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.02729,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 419.539020] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:21.892Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"fb6393f4cd19e3298c8c6e4e","created_at":"2026-10-02T16:24:21.491Z","trace_propagation_headers":{"sentry-trace":"fccc11cd6c5c4d9cb6fab3cf995af8be-942e988359c7489e","baggage":"sentry-trace_id=fccc11cd6c5c4d9cb6fab3cf995af8be,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPWMX36CQH5NJ0WSE420VZ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:21.494Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-fb6393f4cd19e3298c8c6e4e: start","job_status":"start","queue_duration_s":0.39765,"scheduling_latency_s":0.397685} gitlab # [ 419.555631] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:22.070Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"fb6393f4cd19e3298c8c6e4e","created_at":"2026-10-02T16:24:21.491Z","trace_propagation_headers":{"sentry-trace":"fccc11cd6c5c4d9cb6fab3cf995af8be-942e988359c7489e","baggage":"sentry-trace_id=fccc11cd6c5c4d9cb6fab3cf995af8be,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YPWMX36CQH5NJ0WSE420VZ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:21.494Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-fb6393f4cd19e3298c8c6e4e: done: 0.178574 sec","job_status":"done","queue_duration_s":0.39765,"scheduling_latency_s":0.397685,"gitaly_calls":2,"gitaly_duration_s":0.007582,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.008349,"redis_read_bytes":79,"redis_write_bytes":5122,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.00125,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001059,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000429,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002378,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000249,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000436,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001298,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.140534,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.1314887,"duration_s":0.178574,"completed_at":"2026-10-02T16:24:22.070Z","load_balancing_strategy":"primary","db_duration_s":0.031872,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 419.609892] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:22.072Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"f23a90cf8d00307e3377a6d9","created_at":"2026-10-02T16:24:21.978Z","trace_propagation_headers":{"sentry-trace":"fccc11cd6c5c4d9cb6fab3cf995af8be-6a98f10ec3ec4824","baggage":"sentry-trace_id=fccc11cd6c5c4d9cb6fab3cf995af8be,sentry-release=Unknown"},"correlation_id":"01M3YPWMX36CQH5NJ0WSE420VZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:21.982Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-f23a90cf8d00307e3377a6d9: start","job_status":"start","queue_duration_s":0.090174,"scheduling_latency_s":0.090194} gitlab # [ 419.626873] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:22.117Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"f23a90cf8d00307e3377a6d9","created_at":"2026-10-02T16:24:21.978Z","trace_propagation_headers":{"sentry-trace":"fccc11cd6c5c4d9cb6fab3cf995af8be-6a98f10ec3ec4824","baggage":"sentry-trace_id=fccc11cd6c5c4d9cb6fab3cf995af8be,sentry-release=Unknown"},"correlation_id":"01M3YPWMX36CQH5NJ0WSE420VZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:21.982Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-f23a90cf8d00307e3377a6d9: done: 0.044954 sec","job_status":"done","queue_duration_s":0.090174,"scheduling_latency_s":0.090194,"redis_calls":7,"redis_duration_s":0.001491,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001112,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000379,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034991,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.044954,"completed_at":"2026-10-02T16:24:22.117Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005851,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses gitlab # [ 419.841890] gitaly[948]: time="2026-10-02T16:24:22.692Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWPB032DV698R3WEE684D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:24:52.190" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:22.690" grpc.time_ms=0.884 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses, in 0.16 seconds) gitlab # [ 419.876803] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:24:22 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 101 gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses gitlab # [ 420.981967] gitaly[948]: time="2026-10-02T16:24:23.831Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWQEGDM671DDJ0XJS7737 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:24:53.329" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:23.830" grpc.time_ms=1.086 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses, in 0.14 seconds) gitlab # [ 421.018109] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:24:23 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 106 gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses gitlab # [ 422.124006] gitaly[948]: time="2026-10-02T16:24:24.973Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWRJ8D1NGG16Q3GC9EPZC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:24:54.471" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:24.972" grpc.time_ms=0.833 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses, in 0.14 seconds) gitlab # [ 422.156602] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:24:25 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 101 gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses gitlab # [ 423.265279] gitaly[948]: time="2026-10-02T16:24:26.114Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWSNRRAS04QCECKYEN6FQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:24:56.113" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:26.113" grpc.time_ms=0.812 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses, in 0.15 seconds) gitlab # [ 423.306723] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:24:26 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 115 gitlab # [ 423.820642] nix-daemon[3748]: accepted connection from pid 4870, user nixbot gitlab # [ 423.839282] nix-daemon[3748]: accepted connection from pid 4833, user nixbot gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses gitlab # [ 424.408564] gitaly[948]: time="2026-10-02T16:24:27.258Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWTSNVE6W4R0KNMM0M2VK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:24:57.256" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:27.256" grpc.time_ms=0.902 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses, in 0.14 seconds) gitlab # [ 424.443173] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:24:27 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 102 gitlab # [ 424.614861] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:27.464Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"2db842fc286c1df04e15fc95","created_at":"2026-10-02T16:24:27.444Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7a3f6710135f02500fc2307e34ae2cfd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.461Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-2db842fc286c1df04e15fc95: start","job_status":"start","queue_duration_s":0.002574,"scheduling_latency_s":0.002609} gitlab # [ 424.649362] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:27.499Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 424.669841] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:27.519Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"2db842fc286c1df04e15fc95","created_at":"2026-10-02T16:24:27.444Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7a3f6710135f02500fc2307e34ae2cfd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.461Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-2db842fc286c1df04e15fc95: done: 0.055386 sec","job_status":"done","queue_duration_s":0.002574,"scheduling_latency_s":0.002609,"redis_calls":9,"redis_duration_s":0.011421,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001657,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006668,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003096,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019103,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.055386,"completed_at":"2026-10-02T16:24:27.519Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002162,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 424.735332] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:27.521Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"cd5412c698d603d34cd625a7","created_at":"2026-10-02T16:24:27.469Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"3c741d41fb899accceed07fee58981b6","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.484Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-cd5412c698d603d34cd625a7: start","job_status":"start","queue_duration_s":0.037762,"scheduling_latency_s":0.037793} gitlab # [ 424.752509] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:27.571Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"cd5412c698d603d34cd625a7","created_at":"2026-10-02T16:24:27.469Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"3c741d41fb899accceed07fee58981b6","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.484Z","job_size_bytes":3,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-cd5412c698d603d34cd625a7: done: 0.050145 sec","job_status":"done","queue_duration_s":0.037762,"scheduling_latency_s":0.037793,"redis_calls":10,"redis_duration_s":0.01081,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001041,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.008831,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000938,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019134,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00519122,"duration_s":0.050145,"completed_at":"2026-10-02T16:24:27.571Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001992,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 424.802361] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:27.576Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"abce73d33aa64052159c3756","created_at":"2026-10-02T16:24:27.497Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9ff3bc5a4db1a585b66823d794db717c","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.545Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-abce73d33aa64052159c3756: start","job_status":"start","queue_duration_s":0.031074,"scheduling_latency_s":0.031102} gitlab # [ 424.817527] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:27.663Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"abce73d33aa64052159c3756","created_at":"2026-10-02T16:24:27.497Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9ff3bc5a4db1a585b66823d794db717c","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.545Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-abce73d33aa64052159c3756: done: 0.086307 sec","job_status":"done","queue_duration_s":0.031074,"scheduling_latency_s":0.031102,"redis_calls":12,"redis_duration_s":0.017841,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002733,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.010817,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004291,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031976,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01474284,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.086307,"completed_at":"2026-10-02T16:24:27.662Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007864,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 424.862920] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:27.669Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"a11cdcaac273eefeb213a3e6","created_at":"2026-10-02T16:24:27.559Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1855c64d1a480fa2d02f973b8a043c4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.574Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-a11cdcaac273eefeb213a3e6: start","job_status":"start","queue_duration_s":0.094898,"scheduling_latency_s":0.094925} gitlab # [ 424.878023] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:27.722Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 424.899006] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:27.748Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"a11cdcaac273eefeb213a3e6","created_at":"2026-10-02T16:24:27.559Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1855c64d1a480fa2d02f973b8a043c4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.574Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-a11cdcaac273eefeb213a3e6: done: 0.079557 sec","job_status":"done","queue_duration_s":0.094898,"scheduling_latency_s":0.094925,"redis_calls":9,"redis_duration_s":0.020145,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002568,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006735,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.010842,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018167,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.079557,"completed_at":"2026-10-02T16:24:27.748Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000721,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 424.942996] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:27.754Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"3c247a890641ccbdcf053bdd","created_at":"2026-10-02T16:24:27.582Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f4cb39b45f77f63af30b666d6a3dd8d9","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.590Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-3c247a890641ccbdcf053bdd: start","job_status":"start","queue_duration_s":0.163409,"scheduling_latency_s":0.163439} gitlab # [ 425.021300] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:27.870Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"3c247a890641ccbdcf053bdd","created_at":"2026-10-02T16:24:27.582Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f4cb39b45f77f63af30b666d6a3dd8d9","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.590Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-3c247a890641ccbdcf053bdd: done: 0.116878 sec","job_status":"done","queue_duration_s":0.163409,"scheduling_latency_s":0.163439,"redis_calls":9,"redis_duration_s":0.010918,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001657,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006506,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002755,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029672,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.116878,"completed_at":"2026-10-02T16:24:27.870Z","load_balancing_strategy":"primary","db_duration_s":0.018172,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.081976] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:27.877Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"d5ec897cd1cb19c80a3ed2d5","created_at":"2026-10-02T16:24:27.594Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b50e83f1cb9a236edfa0c9fd2f4f1655","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.610Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-d5ec897cd1cb19c80a3ed2d5: start","job_status":"start","queue_duration_s":0.266291,"scheduling_latency_s":0.266325} gitlab # [ 425.097773] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:27.928Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 425.101108] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:27.948Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"d5ec897cd1cb19c80a3ed2d5","created_at":"2026-10-02T16:24:27.594Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b50e83f1cb9a236edfa0c9fd2f4f1655","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.610Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-d5ec897cd1cb19c80a3ed2d5: done: 0.07186 sec","job_status":"done","queue_duration_s":0.266291,"scheduling_latency_s":0.266325,"redis_calls":9,"redis_duration_s":0.011292,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001842,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005801,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003649,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0193,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.07186,"completed_at":"2026-10-02T16:24:27.948Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005884,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.285366] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:28.135Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"28c96d2405aa1053d0bdc216","created_at":"2026-10-02T16:24:27.631Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0503b2e0b11ba8604c7d8ddcd44750cc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.643Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-28c96d2405aa1053d0bdc216: start","job_status":"start","queue_duration_s":0.491441,"scheduling_latency_s":0.491476} gitlab # [ 425.379774] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:28.227Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"28c96d2405aa1053d0bdc216","created_at":"2026-10-02T16:24:27.631Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0503b2e0b11ba8604c7d8ddcd44750cc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.643Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-28c96d2405aa1053d0bdc216: done: 0.092826 sec","job_status":"done","queue_duration_s":0.491441,"scheduling_latency_s":0.491476,"redis_calls":11,"redis_duration_s":0.011562,"redis_read_bytes":9,"redis_write_bytes":1386,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.004748,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004597,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002217,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024543,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.092826,"completed_at":"2026-10-02T16:24:28.227Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0097,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses gitlab # [ 425.445228] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:28.233Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"528870a12f08c0e1132bb7a6","created_at":"2026-10-02T16:24:27.656Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"81fabd1e211d238bbbb5467ba180a432","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.667Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-528870a12f08c0e1132bb7a6: start","job_status":"start","queue_duration_s":0.565749,"scheduling_latency_s":0.565783} gitlab # [ 425.549457] gitaly[948]: time="2026-10-02T16:24:28.399Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWVXBRFMZG46QBDCVF8GX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:24:58.097" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:28.397" grpc.time_ms=0.929 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses, in 0.14 seconds) gitlab # [ 425.565766] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:28.410Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"528870a12f08c0e1132bb7a6","created_at":"2026-10-02T16:24:27.656Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"81fabd1e211d238bbbb5467ba180a432","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.667Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-528870a12f08c0e1132bb7a6: done: 0.176989 sec","job_status":"done","queue_duration_s":0.565749,"scheduling_latency_s":0.565783,"redis_calls":13,"redis_duration_s":0.013354000000000001,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.005798,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004492,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003064,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.044,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.032,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.072377,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00131954,"exclusive_lock_hold_duration_s":0.08640694,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.176989,"completed_at":"2026-10-02T16:24:28.410Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.054432,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.616007] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:28.423Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"ca5a9c339408398ade294184","created_at":"2026-10-02T16:24:27.683Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"52d6707f8cb89477f70efe91622ab21f","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.691Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-ca5a9c339408398ade294184: start","job_status":"start","queue_duration_s":0.73124,"scheduling_latency_s":0.731271} gitlab # [ 425.630531] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:28.463Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"ca5a9c339408398ade294184","created_at":"2026-10-02T16:24:27.683Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"52d6707f8cb89477f70efe91622ab21f","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.691Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-ca5a9c339408398ade294184: done: 0.040394 sec","job_status":"done","queue_duration_s":0.73124,"scheduling_latency_s":0.731271,"redis_calls":9,"redis_duration_s":0.0018830000000000001,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000437,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001038,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000408,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023301,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040394,"completed_at":"2026-10-02T16:24:28.463Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005141,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.674647] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:24:28 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 100 gitlab # [ 425.677556] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:28.464Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"5aaf255d9eea193c60e7279e","created_at":"2026-10-02T16:24:27.701Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"94562195522440df9f369a5f9faf29a3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.716Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-5aaf255d9eea193c60e7279e: start","job_status":"start","queue_duration_s":0.748071,"scheduling_latency_s":0.748096} gitlab # [ 425.691562] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:28.500Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"5aaf255d9eea193c60e7279e","created_at":"2026-10-02T16:24:27.701Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"94562195522440df9f369a5f9faf29a3","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.716Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-5aaf255d9eea193c60e7279e: done: 0.035435 sec","job_status":"done","queue_duration_s":0.748071,"scheduling_latency_s":0.748096,"redis_calls":11,"redis_duration_s":0.002794,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000727,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001077,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00099,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020397,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00066794,"exclusive_lock_hold_duration_s":0.00844714,"duration_s":0.035435,"completed_at":"2026-10-02T16:24:28.500Z","load_balancing_strategy":"primary","db_duration_s":0.004052,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.734401] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:28.501Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"32692a9321f5aa97a1fb9ccc","created_at":"2026-10-02T16:24:27.736Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d2bbdd33a516151aac8547e7dc21f118","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T16:24:27.752Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-32692a9321f5aa97a1fb9ccc: start","job_status":"start","queue_duration_s":0.749118,"scheduling_latency_s":0.749136} gitlab # [ 425.748397] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:28.515Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 425.751757] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:28.526Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"32692a9321f5aa97a1fb9ccc","created_at":"2026-10-02T16:24:27.736Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d2bbdd33a516151aac8547e7dc21f118","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T16:24:27.752Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-32692a9321f5aa97a1fb9ccc: done: 0.024362 sec","job_status":"done","queue_duration_s":0.749118,"scheduling_latency_s":0.749136,"redis_calls":9,"redis_duration_s":0.00169,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00038,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00094,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00037,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013379,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024362,"completed_at":"2026-10-02T16:24:28.526Z","load_balancing_strategy":"primary","db_duration_s":0.000679,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.795319] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:28.527Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Environments::AutoStopCronWorker","jid":"c3f9b2fecd817f01bcc292c9","created_at":"2026-10-02T16:24:27.766Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee4f4bb77f3db34e67d851e2a445288e","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_delivery","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:037d6a92006e3beb0e73d758a3f9a2a2a104eaa8b9d3c2b1a2cf37915906a513","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.775Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Environments::AutoStopCronWorker JID-c3f9b2fecd817f01bcc292c9: start","job_status":"start","queue_duration_s":0.752271,"scheduling_latency_s":0.752289} gitlab # [ 425.809213] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:28.559Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Environments::AutoStopCronWorker","jid":"c3f9b2fecd817f01bcc292c9","created_at":"2026-10-02T16:24:27.766Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee4f4bb77f3db34e67d851e2a445288e","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_delivery","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:037d6a92006e3beb0e73d758a3f9a2a2a104eaa8b9d3c2b1a2cf37915906a513","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.775Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Environments::AutoStopCronWorker JID-c3f9b2fecd817f01bcc292c9: done: 0.031667 sec","job_status":"done","queue_duration_s":0.752271,"scheduling_latency_s":0.752289,"redis_calls":12,"redis_duration_s":0.0022429999999999998,"redis_read_bytes":11,"redis_write_bytes":1368,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000384,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001004,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000855,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":503,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017392,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0004896,"exclusive_lock_hold_duration_s":0.00992624,"duration_s":0.031667,"completed_at":"2026-10-02T16:24:28.559Z","load_balancing_strategy":"primary","db_duration_s":0.005168,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.852440] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:28.560Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"f6ce293538c6af98a5fd49c4","created_at":"2026-10-02T16:24:27.792Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d5bedcee49211a0807ba0d5e09aaed1b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.810Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-f6ce293538c6af98a5fd49c4: start","job_status":"start","queue_duration_s":0.749648,"scheduling_latency_s":0.749665} gitlab # [ 425.874099] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:28.581Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"f6ce293538c6af98a5fd49c4","created_at":"2026-10-02T16:24:27.792Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d5bedcee49211a0807ba0d5e09aaed1b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.810Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-f6ce293538c6af98a5fd49c4: done: 0.020644 sec","job_status":"done","queue_duration_s":0.749648,"scheduling_latency_s":0.749665,"redis_calls":9,"redis_duration_s":0.001622,"redis_read_bytes":7,"redis_write_bytes":1192,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000371,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000346,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011593,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020644,"completed_at":"2026-10-02T16:24:28.581Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000578,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.924062] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:28.582Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"c87c1e78242ee2e880ef5f6b","created_at":"2026-10-02T16:24:27.832Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b1860f4f7c1bc12844c6325ec4a55e18","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.844Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-c87c1e78242ee2e880ef5f6b: start","job_status":"start","queue_duration_s":0.738371,"scheduling_latency_s":0.738388} gitlab # [ 425.938679] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:28.611Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"c87c1e78242ee2e880ef5f6b","created_at":"2026-10-02T16:24:27.832Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b1860f4f7c1bc12844c6325ec4a55e18","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.844Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-c87c1e78242ee2e880ef5f6b: done: 0.02919 sec","job_status":"done","queue_duration_s":0.738371,"scheduling_latency_s":0.738388,"redis_calls":9,"redis_duration_s":0.001731,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000394,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000968,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000369,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016059,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02919,"completed_at":"2026-10-02T16:24:28.611Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003009,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.981889] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:28.613Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"5de3dda5bf1af7cad54a6be0","created_at":"2026-10-02T16:24:27.858Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ddbd7bc3135b17b9cdb185f207342b44","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T16:24:27.875Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-5de3dda5bf1af7cad54a6be0: start","job_status":"start","queue_duration_s":0.737831,"scheduling_latency_s":0.737848} gitlab # [ 425.996501] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:28.647Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"5de3dda5bf1af7cad54a6be0","created_at":"2026-10-02T16:24:27.858Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ddbd7bc3135b17b9cdb185f207342b44","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T16:24:27.875Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-5de3dda5bf1af7cad54a6be0: done: 0.034194 sec","job_status":"done","queue_duration_s":0.737831,"scheduling_latency_s":0.737848,"redis_calls":9,"redis_duration_s":0.0018020000000000002,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000408,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001012,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000382,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034194,"completed_at":"2026-10-02T16:24:28.647Z","load_balancing_strategy":"primary","db_duration_s":0.004026,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 426.038677] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:28.649Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"23d8d804d5928224ad819f7a","created_at":"2026-10-02T16:24:27.895Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea432929ed3bdc3db10d8164d7e2d05e","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.909Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-23d8d804d5928224ad819f7a: start","job_status":"start","queue_duration_s":0.739669,"scheduling_latency_s":0.739695} gitlab # [ 426.053848] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:28.687Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"23d8d804d5928224ad819f7a","created_at":"2026-10-02T16:24:27.895Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea432929ed3bdc3db10d8164d7e2d05e","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.909Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-23d8d804d5928224ad819f7a: done: 0.037536 sec","job_status":"done","queue_duration_s":0.739669,"scheduling_latency_s":0.739695,"redis_calls":11,"redis_duration_s":0.002754,"redis_read_bytes":9,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00042,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001199,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001135,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02209,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00075776,"exclusive_lock_hold_duration_s":0.00598512,"duration_s":0.037536,"completed_at":"2026-10-02T16:24:28.686Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003246,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 426.097749] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:28.688Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"5e92bb12127e3c885a41108e","created_at":"2026-10-02T16:24:27.927Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"98c394d8c5396183320b93dd0fe01360","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.945Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-5e92bb12127e3c885a41108e: start","job_status":"start","queue_duration_s":0.743671,"scheduling_latency_s":0.743692} gitlab # [ 426.112200] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:28.950Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"5e92bb12127e3c885a41108e","created_at":"2026-10-02T16:24:27.927Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"98c394d8c5396183320b93dd0fe01360","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:27.945Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-5e92bb12127e3c885a41108e: done: 0.261716 sec","job_status":"done","queue_duration_s":0.743671,"scheduling_latency_s":0.743692,"redis_calls":13,"redis_duration_s":0.0030229999999999996,"redis_read_bytes":11,"redis_write_bytes":1567,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000833,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00121,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00098,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.034,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.215786,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0006401,"exclusive_lock_hold_duration_s":0.22893468,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.261716,"completed_at":"2026-10-02T16:24:28.950Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.050819,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 426.158567] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:28.952Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["[FILTERED]"],"class":"Database::BackgroundOperation::CronEnqueueWorker","jid":"e9e8b549608861ac0e43a655","created_at":"2026-10-02T16:24:28.131Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f9cde131c405814b3902f75612c17606","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5dfc48484ff3623d3ff52edf287e601d22e2b83c1335126c2dbc7ea70ce18136","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:28.140Z","job_size_bytes":92,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CronEnqueueWorker JID-e9e8b549608861ac0e43a655: start","job_status":"start","queue_duration_s":0.811232,"scheduling_latency_s":0.811252} gitlab # [ 426.173508] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.009Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["[FILTERED]"],"class":"Database::BackgroundOperation::CronEnqueueWorker","jid":"e9e8b549608861ac0e43a655","created_at":"2026-10-02T16:24:28.131Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f9cde131c405814b3902f75612c17606","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5dfc48484ff3623d3ff52edf287e601d22e2b83c1335126c2dbc7ea70ce18136","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:28.140Z","job_size_bytes":92,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CronEnqueueWorker JID-e9e8b549608861ac0e43a655: done: 0.057739 sec","job_status":"done","queue_duration_s":0.811232,"scheduling_latency_s":0.811252,"redis_calls":10,"redis_duration_s":0.0018830000000000001,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000357,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001137,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000389,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038929,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028088,"duration_s":0.057739,"completed_at":"2026-10-02T16:24:29.009Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012428,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 426.218404] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.011Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"67f0b1c32e8251f92e17e640","created_at":"2026-10-02T16:24:28.157Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"89d7f9eafea22ac338e765ec4e23f2f3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T16:24:28.169Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-67f0b1c32e8251f92e17e640: start","job_status":"start","queue_duration_s":0.841977,"scheduling_latency_s":0.841997} gitlab # [ 426.232157] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.024Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 426.235804] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.032Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"67f0b1c32e8251f92e17e640","created_at":"2026-10-02T16:24:28.157Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"89d7f9eafea22ac338e765ec4e23f2f3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T16:24:28.169Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-67f0b1c32e8251f92e17e640: done: 0.021605 sec","job_status":"done","queue_duration_s":0.841977,"scheduling_latency_s":0.841997,"redis_calls":9,"redis_duration_s":0.0016719999999999999,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000347,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000953,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000372,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012542,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021605,"completed_at":"2026-10-02T16:24:29.032Z","load_balancing_strategy":"primary","db_duration_s":0.000558,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 426.278766] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.034Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"0af2174933f5b01a3f474ebd","created_at":"2026-10-02T16:24:28.181Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"707739d7cfa06e63fc047f975eb89e71","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:28.194Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-0af2174933f5b01a3f474ebd: start","job_status":"start","queue_duration_s":0.839555,"scheduling_latency_s":0.839596} gitlab # [ 426.292645] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.054Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"0af2174933f5b01a3f474ebd","created_at":"2026-10-02T16:24:28.181Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"707739d7cfa06e63fc047f975eb89e71","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:28.194Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-0af2174933f5b01a3f474ebd: done: 0.020319 sec","job_status":"done","queue_duration_s":0.839555,"scheduling_latency_s":0.839596,"redis_calls":11,"redis_duration_s":0.00213,"redis_read_bytes":8,"redis_write_bytes":1094,"redis_cache_calls":2,"redis_cache_duration_s":0.000413,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000392,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00095,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":482,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000375,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012445,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020319,"completed_at":"2026-10-02T16:24:29.054Z","load_balancing_strategy":"primary","db_duration_s":0.000408,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 426.335813] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.056Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"7b634db83ee4adfc37962d1a","created_at":"2026-10-02T16:24:28.213Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7161b7cdf123e26fd8f627b8602fd04c","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:28.225Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-7b634db83ee4adfc37962d1a: start","job_status":"start","queue_duration_s":0.83058,"scheduling_latency_s":0.830599} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses gitlab # [ 426.552279] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.402Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"7b634db83ee4adfc37962d1a","created_at":"2026-10-02T16:24:28.213Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7161b7cdf123e26fd8f627b8602fd04c","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:28.225Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-7b634db83ee4adfc37962d1a: done: 0.345624 sec","job_status":"done","queue_duration_s":0.83058,"scheduling_latency_s":0.830599,"redis_calls":564,"redis_duration_s":0.070211,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000328,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000947,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.068936,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.312786,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.345624,"completed_at":"2026-10-02T16:24:29.401Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000454,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 426.617438] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.403Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"2de2aa95d95445d0e3e03a79","created_at":"2026-10-02T16:24:28.240Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"593f9ad808c48880c73b0149040d4a77","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:28.247Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-2de2aa95d95445d0e3e03a79: start","job_status":"start","queue_duration_s":1.156641,"scheduling_latency_s":1.156668} gitlab # [ 426.637003] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.430Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"2de2aa95d95445d0e3e03a79","created_at":"2026-10-02T16:24:28.240Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"593f9ad808c48880c73b0149040d4a77","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:28.247Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-2de2aa95d95445d0e3e03a79: done: 0.026455 sec","job_status":"done","queue_duration_s":1.156641,"scheduling_latency_s":1.156668,"redis_calls":11,"redis_duration_s":0.0024879999999999998,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000455,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000991,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001042,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015659,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00055928,"exclusive_lock_hold_duration_s":0.00342404,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.026455,"completed_at":"2026-10-02T16:24:29.430Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002109,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 426.684261] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.431Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"18adb0b6c1b406b66e8b1d91","created_at":"2026-10-02T16:24:28.252Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:28.266Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-18adb0b6c1b406b66e8b1d91: start","job_status":"start","queue_duration_s":1.165138,"scheduling_latency_s":1.165156} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses, in 0.14 seconds) gitlab # [ 426.698080] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.547Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"18adb0b6c1b406b66e8b1d91","created_at":"2026-10-02T16:24:28.252Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:28.266Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-18adb0b6c1b406b66e8b1d91: done: 0.115406 sec","job_status":"done","queue_duration_s":1.165138,"scheduling_latency_s":1.165156,"redis_calls":70,"redis_duration_s":0.011609,"redis_read_bytes":118,"redis_write_bytes":42580,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.00192,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000555,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.008641,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000493,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.101332,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.115406,"completed_at":"2026-10-02T16:24:29.546Z","load_balancing_strategy":"primary","db_duration_s":0.002253,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 426.742170] gitaly[948]: time="2026-10-02T16:24:29.545Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWX17HPBRC71Y9NMJTHPE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:24:59.043" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:29.544" grpc.time_ms=0.825 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.755650] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:24:29 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 93 gitlab # [ 426.758607] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.548Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"549623cf710909074f819654","created_at":"2026-10-02T16:24:28.285Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"775937ba1dae6bf03fbcb16de9392d53","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:28.299Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-549623cf710909074f819654: start","job_status":"start","queue_duration_s":1.249043,"scheduling_latency_s":1.249065} gitlab # [ 426.772599] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.588Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"549623cf710909074f819654","created_at":"2026-10-02T16:24:28.285Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"775937ba1dae6bf03fbcb16de9392d53","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:28.299Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-549623cf710909074f819654: done: 0.040004 sec","job_status":"done","queue_duration_s":1.249043,"scheduling_latency_s":1.249065,"redis_calls":18,"redis_duration_s":0.00293,"redis_read_bytes":20,"redis_write_bytes":4702,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000608,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000254,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001532,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000536,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027916,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040004,"completed_at":"2026-10-02T16:24:29.588Z","load_balancing_strategy":"primary","db_duration_s":0.002399,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 426.817701] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.589Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"28725b3ceae886a19b61e818","created_at":"2026-10-02T16:24:28.327Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"078b5c9ec60db427bd2bf44f1ffd58e2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:28.336Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-28725b3ceae886a19b61e818: start","job_status":"start","queue_duration_s":1.252846,"scheduling_latency_s":1.252864} gitlab # [ 426.832857] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.637Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"28725b3ceae886a19b61e818","created_at":"2026-10-02T16:24:28.327Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"078b5c9ec60db427bd2bf44f1ffd58e2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:28.336Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-28725b3ceae886a19b61e818: done: 0.047406 sec","job_status":"done","queue_duration_s":1.252846,"scheduling_latency_s":1.252864,"redis_calls":11,"redis_duration_s":0.0028759999999999997,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000394,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001001,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001481,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024485,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00063914,"exclusive_lock_hold_duration_s":0.01509932,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.047406,"completed_at":"2026-10-02T16:24:29.637Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014208,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 426.878684] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.638Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"12960b2fca2eeeb0da760c6a","created_at":"2026-10-02T16:24:28.343Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"59a0f40849f3e19d2cec2f865bc663e4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:28.356Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-12960b2fca2eeeb0da760c6a: start","job_status":"start","queue_duration_s":1.282408,"scheduling_latency_s":1.282428} gitlab # [ 426.893792] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.653Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 426.897277] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.665Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"12960b2fca2eeeb0da760c6a","created_at":"2026-10-02T16:24:28.343Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"59a0f40849f3e19d2cec2f865bc663e4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:28.356Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-12960b2fca2eeeb0da760c6a: done: 0.026965 sec","job_status":"done","queue_duration_s":1.282408,"scheduling_latency_s":1.282428,"redis_calls":9,"redis_duration_s":0.00167,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000414,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000909,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000347,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015035,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026965,"completed_at":"2026-10-02T16:24:29.665Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000487,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 426.940751] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.667Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"cc224bdcedf51fc5cb9da30f","created_at":"2026-10-02T16:24:28.369Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8377856943e46ce4c9551567803f0d3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:28.385Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-cc224bdcedf51fc5cb9da30f: start","job_status":"start","queue_duration_s":1.281193,"scheduling_latency_s":1.281209} gitlab # [ 426.955591] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.691Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"cc224bdcedf51fc5cb9da30f","created_at":"2026-10-02T16:24:28.369Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8377856943e46ce4c9551567803f0d3e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:28.385Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-cc224bdcedf51fc5cb9da30f: done: 0.024383 sec","job_status":"done","queue_duration_s":1.281193,"scheduling_latency_s":1.281209,"redis_calls":12,"redis_duration_s":0.002127,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000374,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001044,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000709,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013391,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026052,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.024383,"completed_at":"2026-10-02T16:24:29.691Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000412,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.002284] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.693Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"8ba48fd0d99eacf8f72ca00f","created_at":"2026-10-02T16:24:28.404Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee384595796f457a0b57acf9786d8ba0","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:28.416Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-8ba48fd0d99eacf8f72ca00f: start","job_status":"start","queue_duration_s":1.276818,"scheduling_latency_s":1.276834} gitlab # [ 427.018372] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.725Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"8ba48fd0d99eacf8f72ca00f","created_at":"2026-10-02T16:24:28.404Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-616583fbebd54145","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee384595796f457a0b57acf9786d8ba0","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-02T16:24:28.416Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-8ba48fd0d99eacf8f72ca00f: done: 0.032183 sec","job_status":"done","queue_duration_s":1.276818,"scheduling_latency_s":1.276834,"redis_calls":9,"redis_duration_s":0.002208,"redis_read_bytes":7,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000401,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001425,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000382,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019483,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032183,"completed_at":"2026-10-02T16:24:29.725Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003763,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.061937] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.726Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bbee98ed3bbafcf2191a846a","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bbee98ed3bbafcf2191a846a: start","job_status":"start","queue_duration_s":0.190081,"scheduling_latency_s":0.190098} gitlab # [ 427.074865] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.751Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bbee98ed3bbafcf2191a846a","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bbee98ed3bbafcf2191a846a: done: 0.024352 sec","job_status":"done","queue_duration_s":0.190081,"scheduling_latency_s":0.190098,"redis_calls":15,"redis_duration_s":0.002781,"redis_read_bytes":12,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00053,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000731,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00152,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015021,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024352,"completed_at":"2026-10-02T16:24:29.751Z","load_balancing_strategy":"primary","db_duration_s":0.000515,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.117616] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.752Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97d3fc30946155470d873d8b","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97d3fc30946155470d873d8b: start","job_status":"start","queue_duration_s":0.215933,"scheduling_latency_s":0.21595} gitlab # [ 427.130617] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.768Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97d3fc30946155470d873d8b","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97d3fc30946155470d873d8b: done: 0.016107 sec","job_status":"done","queue_duration_s":0.215933,"scheduling_latency_s":0.21595,"redis_calls":13,"redis_duration_s":0.002022,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000709,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001313,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01099,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016107,"completed_at":"2026-10-02T16:24:29.768Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.171040] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.770Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84a8558b7912d27c5540480d","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84a8558b7912d27c5540480d: start","job_status":"start","queue_duration_s":0.23365,"scheduling_latency_s":0.233666} gitlab # [ 427.183940] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.786Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84a8558b7912d27c5540480d","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84a8558b7912d27c5540480d: done: 0.015711 sec","job_status":"done","queue_duration_s":0.23365,"scheduling_latency_s":0.233666,"redis_calls":13,"redis_duration_s":0.002013,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000687,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001326,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010737,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015711,"completed_at":"2026-10-02T16:24:29.786Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.225607] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.787Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7063ca260f1da92b5f73fa8a","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7063ca260f1da92b5f73fa8a: start","job_status":"start","queue_duration_s":0.251054,"scheduling_latency_s":0.251071} gitlab # [ 427.238512] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.804Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7063ca260f1da92b5f73fa8a","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7063ca260f1da92b5f73fa8a: done: 0.016336 sec","job_status":"done","queue_duration_s":0.251054,"scheduling_latency_s":0.251071,"redis_calls":13,"redis_duration_s":0.001924,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000686,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001238,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011412,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016336,"completed_at":"2026-10-02T16:24:29.804Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.278901] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.805Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab3a58469cd7c605df9576c3","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab3a58469cd7c605df9576c3: start","job_status":"start","queue_duration_s":0.268543,"scheduling_latency_s":0.268558} gitlab # [ 427.291822] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.824Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab3a58469cd7c605df9576c3","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab3a58469cd7c605df9576c3: done: 0.019173 sec","job_status":"done","queue_duration_s":0.268543,"scheduling_latency_s":0.268558,"redis_calls":13,"redis_duration_s":0.002298,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000755,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001543,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013286,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019173,"completed_at":"2026-10-02T16:24:29.824Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.332457] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.826Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d3b962f021370c5d795a9eca","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d3b962f021370c5d795a9eca: start","job_status":"start","queue_duration_s":0.289267,"scheduling_latency_s":0.289289} gitlab # [ 427.345275] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.850Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d3b962f021370c5d795a9eca","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d3b962f021370c5d795a9eca: done: 0.024434 sec","job_status":"done","queue_duration_s":0.289267,"scheduling_latency_s":0.289289,"redis_calls":13,"redis_duration_s":0.002268,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0008,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001468,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017282,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024434,"completed_at":"2026-10-02T16:24:29.850Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.385603] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.852Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2ba511568b626ffca2423f19","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2ba511568b626ffca2423f19: start","job_status":"start","queue_duration_s":0.315401,"scheduling_latency_s":0.315425} gitlab # [ 427.398354] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.873Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2ba511568b626ffca2423f19","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2ba511568b626ffca2423f19: done: 0.021735 sec","job_status":"done","queue_duration_s":0.315401,"scheduling_latency_s":0.315425,"redis_calls":13,"redis_duration_s":0.002809,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001333,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001476,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014349,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021735,"completed_at":"2026-10-02T16:24:29.873Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.439915] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.875Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2db791f724dcb0bcf9ea6535","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2db791f724dcb0bcf9ea6535: start","job_status":"start","queue_duration_s":0.338541,"scheduling_latency_s":0.338559} gitlab # [ 427.452917] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.894Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2db791f724dcb0bcf9ea6535","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2db791f724dcb0bcf9ea6535: done: 0.018594 sec","job_status":"done","queue_duration_s":0.338541,"scheduling_latency_s":0.338559,"redis_calls":13,"redis_duration_s":0.0020020000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000709,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001293,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012243,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018594,"completed_at":"2026-10-02T16:24:29.893Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.493175] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.895Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6551dcfccbd6143bac5e4380","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6551dcfccbd6143bac5e4380: start","job_status":"start","queue_duration_s":0.358453,"scheduling_latency_s":0.35847} gitlab # [ 427.506511] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.912Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6551dcfccbd6143bac5e4380","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6551dcfccbd6143bac5e4380: done: 0.016511 sec","job_status":"done","queue_duration_s":0.358453,"scheduling_latency_s":0.35847,"redis_calls":13,"redis_duration_s":0.0020759999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000724,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001352,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010911,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016511,"completed_at":"2026-10-02T16:24:29.911Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.547015] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.913Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cdf5ddce407cd5184baee494","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cdf5ddce407cd5184baee494: start","job_status":"start","queue_duration_s":0.376547,"scheduling_latency_s":0.376562} gitlab # [ 427.559885] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.928Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cdf5ddce407cd5184baee494","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cdf5ddce407cd5184baee494: done: 0.015082 sec","job_status":"done","queue_duration_s":0.376547,"scheduling_latency_s":0.376562,"redis_calls":13,"redis_duration_s":0.001931,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000665,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001266,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009745,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015082,"completed_at":"2026-10-02T16:24:29.928Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.600572] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.929Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f96c42b13e93f4abad7fd1f","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f96c42b13e93f4abad7fd1f: start","job_status":"start","queue_duration_s":0.392766,"scheduling_latency_s":0.392782} gitlab # [ 427.613206] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.944Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f96c42b13e93f4abad7fd1f","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f96c42b13e93f4abad7fd1f: done: 0.014468 sec","job_status":"done","queue_duration_s":0.392766,"scheduling_latency_s":0.392782,"redis_calls":13,"redis_duration_s":0.001958,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000667,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001291,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00951,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014468,"completed_at":"2026-10-02T16:24:29.944Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.654739] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.945Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ac3bfc08087fc1c02509b61","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ac3bfc08087fc1c02509b61: start","job_status":"start","queue_duration_s":0.408378,"scheduling_latency_s":0.408395} gitlab # [ 427.667612] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.959Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ac3bfc08087fc1c02509b61","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ac3bfc08087fc1c02509b61: done: 0.014506 sec","job_status":"done","queue_duration_s":0.408378,"scheduling_latency_s":0.408395,"redis_calls":13,"redis_duration_s":0.001885,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000652,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001233,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00951,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014506,"completed_at":"2026-10-02T16:24:29.959Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.707846] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.960Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29221d06f60a35da114664ef","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29221d06f60a35da114664ef: start","job_status":"start","queue_duration_s":0.424086,"scheduling_latency_s":0.424102} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses gitlab # [ 427.720753] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.975Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29221d06f60a35da114664ef","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29221d06f60a35da114664ef: done: 0.014894 sec","job_status":"done","queue_duration_s":0.424086,"scheduling_latency_s":0.424102,"redis_calls":13,"redis_duration_s":0.001996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000744,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001252,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009794,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014894,"completed_at":"2026-10-02T16:24:29.975Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.760236] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.977Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a63660d5ade7812398091450","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a63660d5ade7812398091450: start","job_status":"start","queue_duration_s":0.440176,"scheduling_latency_s":0.440193} gitlab # [ 427.773249] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.991Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a63660d5ade7812398091450","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a63660d5ade7812398091450: done: 0.01451 sec","job_status":"done","queue_duration_s":0.440176,"scheduling_latency_s":0.440193,"redis_calls":13,"redis_duration_s":0.0019160000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000682,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001234,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009416,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01451,"completed_at":"2026-10-02T16:24:29.991Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.813345] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:29.993Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fbf44ec011bbae4454ee8df2","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fbf44ec011bbae4454ee8df2: start","job_status":"start","queue_duration_s":0.456292,"scheduling_latency_s":0.456308} gitlab # [ 427.826100] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.007Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fbf44ec011bbae4454ee8df2","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fbf44ec011bbae4454ee8df2: done: 0.01456 sec","job_status":"done","queue_duration_s":0.456292,"scheduling_latency_s":0.456308,"redis_calls":13,"redis_duration_s":0.001932,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00066,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001272,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009336,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01456,"completed_at":"2026-10-02T16:24:30.007Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.867576] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.009Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2744d19c9f726402009d789d","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2744d19c9f726402009d789d: start","job_status":"start","queue_duration_s":0.472107,"scheduling_latency_s":0.472123} gitlab # [ 427.880490] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.023Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2744d19c9f726402009d789d","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2744d19c9f726402009d789d: done: 0.014562 sec","job_status":"done","queue_duration_s":0.472107,"scheduling_latency_s":0.472123,"redis_calls":13,"redis_duration_s":0.001844,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000683,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001161,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009442,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014562,"completed_at":"2026-10-02T16:24:30.023Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.920890] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.024Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea9148d271bbad4fe326c114","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea9148d271bbad4fe326c114: start","job_status":"start","queue_duration_s":0.487875,"scheduling_latency_s":0.487891} gitlab # [ 427.933636] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.039Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea9148d271bbad4fe326c114","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea9148d271bbad4fe326c114: done: 0.014656 sec","job_status":"done","queue_duration_s":0.487875,"scheduling_latency_s":0.487891,"redis_calls":13,"redis_duration_s":0.001938,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000686,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001252,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009486,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014656,"completed_at":"2026-10-02T16:24:30.039Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.973585] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.040Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e0d27d8fabe76f49db97b5d","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e0d27d8fabe76f49db97b5d: start","job_status":"start","queue_duration_s":0.503771,"scheduling_latency_s":0.503786} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses, in 0.27 seconds) gitlab # [ 427.986371] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.055Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e0d27d8fabe76f49db97b5d","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e0d27d8fabe76f49db97b5d: done: 0.014524 sec","job_status":"done","queue_duration_s":0.503771,"scheduling_latency_s":0.503786,"redis_calls":13,"redis_duration_s":0.001794,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000653,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001141,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00944,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014524,"completed_at":"2026-10-02T16:24:30.055Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 428.026192] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.056Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"232eed711926057b0ea5260a","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-232eed711926057b0ea5260a: start","job_status":"start","queue_duration_s":0.519466,"scheduling_latency_s":0.519482} gitlab # [ 428.039359] gitaly[948]: time="2026-10-02T16:24:30.814Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWY586S3WX126CN61QQ6T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:25:00.313" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:30.813" grpc.time_ms=0.882 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 428.054187] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:24:30 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 212 gitlab # [ 428.056949] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.070Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"232eed711926057b0ea5260a","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-232eed711926057b0ea5260a: done: 0.014313 sec","job_status":"done","queue_duration_s":0.519466,"scheduling_latency_s":0.519482,"redis_calls":13,"redis_duration_s":0.001912,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000662,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00125,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0094,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014313,"completed_at":"2026-10-02T16:24:30.070Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 428.096960] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.071Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4c33c313a28072cad246b97","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4c33c313a28072cad246b97: start","job_status":"start","queue_duration_s":0.534975,"scheduling_latency_s":0.534991} gitlab # [ 428.109742] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.086Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4c33c313a28072cad246b97","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4c33c313a28072cad246b97: done: 0.014203 sec","job_status":"done","queue_duration_s":0.534975,"scheduling_latency_s":0.534991,"redis_calls":13,"redis_duration_s":0.0019529999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000663,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00129,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00936,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014203,"completed_at":"2026-10-02T16:24:30.086Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 428.150082] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.087Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1523d7c2c6c31a358ac6ecfd","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1523d7c2c6c31a358ac6ecfd: start","job_status":"start","queue_duration_s":0.550422,"scheduling_latency_s":0.550438} gitlab # [ 428.162925] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.102Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1523d7c2c6c31a358ac6ecfd","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1523d7c2c6c31a358ac6ecfd: done: 0.014792 sec","job_status":"done","queue_duration_s":0.550422,"scheduling_latency_s":0.550438,"redis_calls":13,"redis_duration_s":0.002344,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000997,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001347,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009514,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014792,"completed_at":"2026-10-02T16:24:30.102Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 428.203287] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.103Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9697902d5b7312195be804b6","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9697902d5b7312195be804b6: start","job_status":"start","queue_duration_s":0.566497,"scheduling_latency_s":0.566512} gitlab # [ 428.216200] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.117Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9697902d5b7312195be804b6","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9697902d5b7312195be804b6: done: 0.014273 sec","job_status":"done","queue_duration_s":0.566497,"scheduling_latency_s":0.566512,"redis_calls":13,"redis_duration_s":0.0018830000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000662,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001221,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009417,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014273,"completed_at":"2026-10-02T16:24:30.117Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 428.257867] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.118Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"caa63fcfbfd5f8b77ac27e7d","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-caa63fcfbfd5f8b77ac27e7d: start","job_status":"start","queue_duration_s":0.58192,"scheduling_latency_s":0.581936} gitlab # [ 428.271013] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.132Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"caa63fcfbfd5f8b77ac27e7d","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-caa63fcfbfd5f8b77ac27e7d: done: 0.014098 sec","job_status":"done","queue_duration_s":0.58192,"scheduling_latency_s":0.581936,"redis_calls":13,"redis_duration_s":0.001883,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00065,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001233,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009325,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014098,"completed_at":"2026-10-02T16:24:30.132Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 428.311375] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.134Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"201a6c4e52a64a568b3bf32d","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-201a6c4e52a64a568b3bf32d: start","job_status":"start","queue_duration_s":0.597096,"scheduling_latency_s":0.597112} gitlab # [ 428.324638] nix-daemon[3748]: accepted connection from pid 4833, user nixbot gitlab # [ 428.325629] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.148Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"201a6c4e52a64a568b3bf32d","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-201a6c4e52a64a568b3bf32d: done: 0.014529 sec","job_status":"done","queue_duration_s":0.597096,"scheduling_latency_s":0.597112,"redis_calls":13,"redis_duration_s":0.002051,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000713,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001338,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009495,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014529,"completed_at":"2026-10-02T16:24:30.148Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 428.365623] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.150Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5ce2f0ce68955a8f31cca898","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5ce2f0ce68955a8f31cca898: start","job_status":"start","queue_duration_s":0.61334,"scheduling_latency_s":0.613367} gitlab # [ 428.378503] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.171Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5ce2f0ce68955a8f31cca898","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5ce2f0ce68955a8f31cca898: done: 0.021026 sec","job_status":"done","queue_duration_s":0.61334,"scheduling_latency_s":0.613367,"redis_calls":13,"redis_duration_s":0.002149,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000785,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001364,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021026,"completed_at":"2026-10-02T16:24:30.171Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 428.419067] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.172Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e72b67f2313312a5e9ea5790","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e72b67f2313312a5e9ea5790: start","job_status":"start","queue_duration_s":0.635835,"scheduling_latency_s":0.635857} gitlab # [ 428.431909] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.193Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e72b67f2313312a5e9ea5790","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e72b67f2313312a5e9ea5790: done: 0.020544 sec","job_status":"done","queue_duration_s":0.635835,"scheduling_latency_s":0.635857,"redis_calls":13,"redis_duration_s":0.00237,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000816,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001554,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014188,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020544,"completed_at":"2026-10-02T16:24:30.193Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 428.473157] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.195Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bfbf2e6b68908b56fef0cd12","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bfbf2e6b68908b56fef0cd12: start","job_status":"start","queue_duration_s":0.658465,"scheduling_latency_s":0.658487} gitlab # [ 428.485927] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.216Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bfbf2e6b68908b56fef0cd12","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bfbf2e6b68908b56fef0cd12: done: 0.021048 sec","job_status":"done","queue_duration_s":0.658465,"scheduling_latency_s":0.658487,"redis_calls":13,"redis_duration_s":0.002369,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000814,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001555,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014377,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021048,"completed_at":"2026-10-02T16:24:30.216Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 428.525910] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.217Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9864f4d993e2fcbd7c13f7b5","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9864f4d993e2fcbd7c13f7b5: start","job_status":"start","queue_duration_s":0.681059,"scheduling_latency_s":0.681081} gitlab # [ 428.538617] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.235Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9864f4d993e2fcbd7c13f7b5","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9864f4d993e2fcbd7c13f7b5: done: 0.01791 sec","job_status":"done","queue_duration_s":0.681059,"scheduling_latency_s":0.681081,"redis_calls":13,"redis_duration_s":0.002169,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000752,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001417,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011901,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01791,"completed_at":"2026-10-02T16:24:30.235Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 428.578511] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.237Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11219939f1e8770448ff4f63","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11219939f1e8770448ff4f63: start","job_status":"start","queue_duration_s":0.70031,"scheduling_latency_s":0.70033} gitlab # [ 428.591332] gitaly[948]: time="2026-10-02T16:24:31.341Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWYRTMEWW5A8ZZCSB1JQW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:25:01.040" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:31.340" grpc.time_ms=0.906 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 428.604534] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.253Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11219939f1e8770448ff4f63","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11219939f1e8770448ff4f63: done: 0.015876 sec","job_status":"done","queue_duration_s":0.70031,"scheduling_latency_s":0.70033,"redis_calls":13,"redis_duration_s":0.002005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000741,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001264,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0105,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015876,"completed_at":"2026-10-02T16:24:30.253Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 428.645025] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.254Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef047ffe081c06b1e9ec9e5e","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef047ffe081c06b1e9ec9e5e: start","job_status":"start","queue_duration_s":0.717489,"scheduling_latency_s":0.717538} gitlab # [ 428.658923] gitaly[948]: time="2026-10-02T16:24:31.449Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YPWYRTMEWW5A8ZZCSB1JQW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:24:41.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:24:31.437" grpc.time_ms=11.219 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 428.673276] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.269Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef047ffe081c06b1e9ec9e5e","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef047ffe081c06b1e9ec9e5e: done: 0.014755 sec","job_status":"done","queue_duration_s":0.717489,"scheduling_latency_s":0.717538,"redis_calls":13,"redis_duration_s":0.001853,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000656,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001197,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009673,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014755,"completed_at":"2026-10-02T16:24:30.269Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 428.713551] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.270Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"860b6aa35dd78c3f3b6b7251","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-860b6aa35dd78c3f3b6b7251: start","job_status":"start","queue_duration_s":0.733411,"scheduling_latency_s":0.733431} gitlab # [ 428.726459] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:24:31 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 265 gitlab # [ 428.729380] nixbot[3579]: {"ts": "2026-10-02T16:24:31.526Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a \"HTTP/1.1 201 Created\""} gitlab # [ 428.732241] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.287Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"860b6aa35dd78c3f3b6b7251","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-860b6aa35dd78c3f3b6b7251: done: 0.016983 sec","job_status":"done","queue_duration_s":0.733411,"scheduling_latency_s":0.733431,"redis_calls":13,"redis_duration_s":0.002154,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000816,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001338,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011277,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016983,"completed_at":"2026-10-02T16:24:30.287Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 428.772246] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.288Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5f37471ad4027b1f0b8e1f3","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5f37471ad4027b1f0b8e1f3: start","job_status":"start","queue_duration_s":0.752026,"scheduling_latency_s":0.752054} gitlab # [ 428.784863] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.309Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5f37471ad4027b1f0b8e1f3","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5f37471ad4027b1f0b8e1f3: done: 0.021 sec","job_status":"done","queue_duration_s":0.752026,"scheduling_latency_s":0.752054,"redis_calls":13,"redis_duration_s":0.002301,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000799,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001502,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01399,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021,"completed_at":"2026-10-02T16:24:30.309Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 428.824891] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.311Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0de0dc97fd14ecd1da240e6e","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0de0dc97fd14ecd1da240e6e: start","job_status":"start","queue_duration_s":0.774875,"scheduling_latency_s":0.774904} gitlab # [ 428.837475] gitaly[948]: time="2026-10-02T16:24:31.608Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWZ1CMRFFCCM9ARMBPQBX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:25:01.106" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:24:31.607" grpc.time_ms=0.909 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 428.850498] gitaly[948]: time="2026-10-02T16:24:31.652Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPWYRTMEWW5A8ZZCSB1JQW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:24:31.650" grpc.time_ms=1.677 pid=948 span.kind=server system=grpc gitlab # [ 428.861486] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.331Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0de0dc97fd14ecd1da240e6e","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0de0dc97fd14ecd1da240e6e: done: 0.020141 sec","job_status":"done","queue_duration_s":0.774875,"scheduling_latency_s":0.774904,"redis_calls":13,"redis_duration_s":0.002298,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000785,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001513,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013367,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020141,"completed_at":"2026-10-02T16:24:30.331Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 428.902856] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.333Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c1f40714c13814c5063e2f1","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c1f40714c13814c5063e2f1: start","job_status":"start","queue_duration_s":0.796397,"scheduling_latency_s":0.796417} gitlab # [ 428.915811] gitaly[948]: time="2026-10-02T16:24:31.664Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YPWZ1CMRFFCCM9ARMBPQBX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:24:41.224" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:24:31.655" grpc.time_ms=9.444 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 428.929713] gitaly[948]: time="2026-10-02T16:24:31.685Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPWYRTMEWW5A8ZZCSB1JQW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:24:31.685" grpc.time_ms=0.622 pid=948 span.kind=server system=grpc gitlab # [ 428.939691] nixbot[3579]: {"ts": "2026-10-02T16:24:31.737Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a \"HTTP/1.1 201 Created\""} gitlab # [ 428.942821] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.352Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c1f40714c13814c5063e2f1","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c1f40714c13814c5063e2f1: done: 0.019576 sec","job_status":"done","queue_duration_s":0.796397,"scheduling_latency_s":0.796417,"redis_calls":13,"redis_duration_s":0.002303,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000753,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00155,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013578,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019576,"completed_at":"2026-10-02T16:24:30.352Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 428.983341] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.354Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"604af07590f7fbf2acbb62c3","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-604af07590f7fbf2acbb62c3: start","job_status":"start","queue_duration_s":0.817961,"scheduling_latency_s":0.817985} gitlab # [ 428.996306] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:24:31 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 202 gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses gitlab # [ 428.998886] gitaly[948]: time="2026-10-02T16:24:31.713Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YPWYRTMEWW5A8ZZCSB1JQW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:24:31.712" grpc.time_ms=0.478 pid=948 span.kind=server system=grpc gitlab # [ 429.008551] gitaly[948]: time="2026-10-02T16:24:31.717Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWYRTMEWW5A8ZZCSB1JQW forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T16:25:01.613" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:24:31.614" operations="[repack_refs]" pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.017794] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.376Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"604af07590f7fbf2acbb62c3","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-604af07590f7fbf2acbb62c3: done: 0.021357 sec","job_status":"done","queue_duration_s":0.817961,"scheduling_latency_s":0.817985,"redis_calls":13,"redis_duration_s":0.002545,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000896,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001649,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014402,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021357,"completed_at":"2026-10-02T16:24:30.376Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 429.058258] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.377Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09c1f1ccd232b3e6db20e3ef","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09c1f1ccd232b3e6db20e3ef: start","job_status":"start","queue_duration_s":0.840995,"scheduling_latency_s":0.841017} gitlab # [ 429.073028] gitaly[948]: time="2026-10-02T16:24:31.720Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWYRTMEWW5A8ZZCSB1JQW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T16:25:01.613" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:24:31.614" pid=948 remote_ip="2001:db8:1::1" repository_info="{false {6 703 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-10-02 16:22:37.19974426 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root gitlab # [ 429.085609] gitaly[948]: time="2026-10-02T16:24:31.720Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWYRTMEWW5A8ZZCSB1JQW diskcache=d2c2823e-5bb6-42f1-ae25-d90f322932e6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T16:25:01.613" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:24:31.614" pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.095649] nixbot[3579]: {"ts": "2026-10-02T16:24:31.902Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a \"HTTP/1.1 201 Created\""} gitlab # [ 429.098632] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09c1f1ccd232b3e6db20e3ef","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09c1f1ccd232b3e6db20e3ef: done: 0.019171 sec","job_status":"done","queue_duration_s":0.840995,"scheduling_latency_s":0.841017,"redis_calls":13,"redis_duration_s":0.002255,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000772,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001483,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012854,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019171,"completed_at":"2026-10-02T16:24:30.396Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses, in 0.15 seconds) (finished: subtest: eval and build post commit statuses, in 10.59 seconds) (finished: run the VM test script, in 500.51 seconds) gitlab # [ 429.139339] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.398Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f08453a9ad5f89fc09a3783d","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:24:29.536Z","job_size_bytes":2,"pid":3455,"sidekiq_tid":"m8lf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f08453a9ad5f89fc09a3783d: start","job_status":"start","queue_duration_s":0.861617,"scheduling_latency_s":0.861637} gitlab # [ 429.152760] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:24:31 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 154 gitlab # [ 429.155325] gitlab-workhorse[885]: gitlab 2001:db8:1::1 - - [2026/10/02:16:24:31 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a/statuses HTTP/1.1" 200 1334 "" "curl/8.21.0" 99 gitlab # [ 429.158262] gitaly[948]: time="2026-10-02T16:24:31.720Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=104 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7282 command.oublock=0 command.real_time_ms=101 command.spawn_token_fork_ms=1 command.system_time_ms=73 command.user_time_ms=31 component=gitaly.UnaryServerInterceptor correlation_id=01M3YPWYRTMEWW5A8ZZCSB1JQW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T16:25:01.613" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:24:31.614" grpc.time_ms=106.492 pid=948 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root test script finished in 500.54s cleanup kill QemuMachine (pid 46) gitlab # [ 429.174262] sidekiq-cluster[3455]: {"severity":"INFO","time":"2026-10-02T16:24:30.415Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f08453a9ad5f89fc09a3783d","created_at":"2026-10-02T16:24:29.452Z","trace_propagation_headers":{"sentry-trace":"b3cf03f39e0347c0a06d652252e9cc75-35aeff920ef84dd5","baggage":"sentry-trace_id=b3cf03f39e0347c0a06d652252e9cc75,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"c2190a71ede51dcfc1d33408e4715977","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.clientqemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/41m77i1296n33p6liin8ynr6wh3h6b7m-python3-3.14.6/bin/python3.14) (finished: cleanup, in 1.81 seconds) additionally exposed symbols: gitlab, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-10-02T16:22:48.410Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] [] [] [] [] [] [] [] [] [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-10-02T16:24:21.149187Z', 'started_at': None, 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': {'position': 1, 'blocked_by': []}}] [{'id': 5, 'sha': 'fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T16:24:21.484Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T16:24:21.484Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T16:24:21.484Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T16:24:21.484Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T16:24:21.484Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T16:24:21.484Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T16:24:21.484Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T16:24:21.484Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T16:24:21.484Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 9s (1 warning)', 'created_at': '2026-10-02T16:24:21.484Z', 'started_at': None, 'finished_at': '2026-10-02T16:24:31.482Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': 'fcd155fee69500a8c3cb7fe8b98ad41b8a8e1c3a', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '0 attributes built', 'created_at': '2026-10-02T16:24:31.702Z', 'started_at': None, 'finished_at': '2026-10-02T16:24:31.871Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]